DANISH ACCREDITATION INSTITUTION

Transcription

1 XXXXXXXX 2016 ENQA REVIEW SELF-ASSESSMENT REPORT JANUARY 2016 DANISH ACCREDITATION INSTITUTION

2 Danmarks Akkrediteringsinstitution Citat tilladt ved kildeangivelse Rapporten kan downloades via hjemmesiden

3 Table of Contents Preface... 5 Self-evaluation outline... 6 Part 1 Management Reflections and Perspectives on the Practice of the... 7 Part 2 Accreditation and Education in Denmark Historical Overview The Accreditation Institution The Accreditation Institution s tasks The organisation of the The Accreditation Council The Accreditation Council as Decision Maker Decision-making Process of the Accreditation Council Outline of the Danish Higher Education System Higher Education Institutions and Study Programmes Qualification Levels in the Danish Higher Education System Part 3 Compliance by the with the European Standards and Guidelines for the External Quality Assurance of Higher Education ESG Part 2 - Compliance by the with the ESG for External Quality Assurance ESG 2.1 Consideration of internal quality assurance ESG 2.2 Designing methodologies fit for purpose ESG 2.3 Implementing processes ESG 2.4 Peer-review experts ESG 2.5 Criteria for outcomes ESG 2.6 Reporting ESG 2.7 Complaints and appeals ESG Part 3 Compliance by The with the ESG for Quality Assurance Agencies ESG 3.1 Activities, policy and processes for quality assurance ESG 3.2 Official Status ESG 3.3 Independence ESG 3.4 Thematic analysis ESG 3.5 Resources ESG 3.6 Internal quality assurance and professional conduct ESG 3.7 Cyclical external review of agencies... 62

4 4

5 Preface This self-evaluation by the (AI) was initiated and commissioned by AI. The evaluation has one main purpose: To examine whether the Accreditation Institution meets the Standards and Guidelines for Quality Assurance in the European Higher Education Area (Approved by the Ministerial Conference in Yerevan, May 2015) This document contains the self-evaluation of the AI, which is part of the application to renew membership of European Association for Quality Assurance in Higher Education (ENQA). It is also forming part of the basis for the application to renew the listing of AI in the European Quality Assurance Register for Higher Education (EQAR). The self-evaluation is submitted with a view to the external review panel paying a site-visit to AI in April The report has been prepared by a project group from AI. Other employees at AI have been involved in respect of relevant topics. Copenhagen, Denmark, 20 January

6 Self-evaluation outline The report is divided into the following parts: Part 1 contains reflections on the present status of AI and discusses a number of the challenges facing AI. Part 2 contains a description of the organisation of AI and the Accreditation Council as well as a description of the higher education system in Denmark. Part 3 contains the self-evaluation part of the report, in which an account is made of how AI is compliant with the European standards and guidelines (ESG). A number of annexes in English have been enclosed with the report to support the reading of the self-evaluation. Other documents have not been translated into English, but will be so on the Panel s request (see also the appended Examples of relevant documents at AI). 6

7 Part 1 Management Reflections and Perspectives on the Practice of the Danish Accreditation Institution 7

8 The Management s reflections This section contains the management s reflections regarding the current status of the Danish Accreditation Institution (AI). The management wishes to present an overview of recent developments in the Danish accreditation system and wishes to highlight some of the challenges in our work and in the sector in general. Fit for purpose and documentation requirements from the institutions The Accreditation Council and the Accreditation Institution was established with the Accreditation Act of 2007, and this also introduced programme accreditation in Denmark. All study programmes across all sectors were to be systematically accredited by two independent, external quality assurance operators (ACE Denmark and EVA). The programme accreditation system contained very detailed, pre-defined criteria, which stemmed from a political desire to check the quality of the individual programmes in the almost entirely publicly funded Danish higher education institutions (HEIs). The programme accreditation system was an ambitious system with fixed standards, very consistent assessments and clear consequences for non-compliant programmes. A refusal of accreditation meant that the programmes could ultimately be closed down. Although this decision was (and is) extremely rare, the actual possibility of receiving a refusal of accreditation forced the HEIs to strengthen the quality assurance of programmes much faster than a system with lesser consequences could have done. Thus, the programme accreditation system of 2007 formed the basis for a less problematic transition to the institutional accreditation system of 2013 in which the responsibility for the quality assurance lies with the HEIs themselves. From the very beginning, the HEIs, and especially the universities, were raising the problematic issue of over-documentation. They felt, somewhat rightly, that the system s benefits regarding equal treatment of the HEIs did not measure up entirely to the costs of a programme accreditation system in which all programmes had to be compliant with the same standards and demands for documentation. The programme accreditation system has contributed to increased awareness of quality assurance in the higher education sector, and this quality assurance methodology has proved to be efficient regarding the systematic approach to quality assurance of the individual programme. This systematic approach, however, only gave little freedom to the institutions and programmes to define their own quality standards and set up their own quality assurance processes. Also, the programme accreditation system had confirmed that the quality of the programmes was indeed sufficiently high, since approximately 85 percent of all accredited programmes in all sectors had received a positive accreditation. On the other hand, the high percentage of positive accreditations indicated a less efficient usage of resources, since much effort and resources had been put into every accreditation, compliant or non-compliant, by both the HEIs and AI. In 2010, the first steps in a system change were made. The intention was to offer greater degrees of freedom to the HEIs. Experience from carrying out programme accreditation formed an important basis for the second version of the Danish accreditation system, which came into 8

9 effect in It was on the basis of experience from this system that AI stressed the importance of a new system with less focus on merely documenting compliance with the pre-defined criteria, and more focus on the HEI s own responsibility for the quality assurance of its programmes through real, day-to-day quality work at the institution. With the introduction of an institutional accreditation system in 2013, responsibility for the quality of programmes was placed with the HEIs. Furthermore, it was decided that all HEI sectors should be accredited under the same legislation and criteria. Thus, the institutional accreditation criteria are now established by the Ministry of Higher Education in such a way that they cover all institutions from the largest universities to the smallest maritime school ships. Through interpretation of the broad criteria, AI aims at being able to honour the diversity of these institutions by taking the specific identity and culture of every institution into account. Since focus is still on accreditation, and thus leads to a decision, the system must also be transparent and explicit regarding the demands imposed on the HEIs. The documentation from the institution that forms the basis of the panel s assessments must be sufficiently thorough and nuanced so that a well-substantiated decision can be made by the Council. HEIs prefer their quality assurance work to be assessed on their own terms, but they are also very focused on consistency, i.e. just and uniform assessments and decisions. This forms a dilemma for AI, which we are trying to resolve by focusing on very well-substantiated and thorough reports, so that the assessments and the basis for our decisions are clear and comparable across institutions and sectors. However, the thorough analyses and reports cannot be carried out without sufficient documentation, and it became clear from the evaluations after the first institutional accreditation round that the documentation requirements are still quite burdensome to the institutions. It is clear that we have yet to find the optimal balance between having sufficient material for the accreditation panel to make sound, fair and nuanced assessments and not requiring too much documentation from the institutions. We are focused on ensuring that the accreditation system remains fit for purpose. Thus, we will continue our on-going dialogue with the institutions at management level. Furthermore, we will continue to carry out thematic analyses that present overviews on subjects that are important to the institutions. The aim of both the dialogue and the thematic analyses is to disseminate knowledge, but also to learn more from the institutions, so that the processes and methods are continuously improved and are as efficient and relevant to the institutions as possible. Development and control We are well aware that a system with a strong element of control such as the accreditation system, while strengthening the possibilities for impact, also demands a very cautious preparation and organization to succeed. The institutional accreditation system aims at supporting quality development at the institutions. This is done by refraining from setting fixed 9

10 and detailed standards for how the institutions are to carry out quality assurance, but rather assessing whether the individual institution s system functions in practice and in its own right. The institutions have a free hand to establish an internal quality assurance system that supports their own goals, as long as they also live up to the accreditation criteria and, thus, the ESG s. As mentioned, the accreditation criteria are very general, so that all institutions can be covered by the same system. As a consequence of this, AI has issued a number of briefs aimed at explaining and interpreting the criteria, so that the institutions are well informed when going into an accreditation process. However, this sort of communication has unfortunately not been received in this spirit by all institutions. Some institutions perceived the briefs as supplementary demands and extra criteria with which the institutions must comply. We are not discouraged by this, since we consider that we are obliged to inform the institutions on a variety of platforms. Still, we find that the reactions are an indication of two issues that we have to address continuously. Firstly, the institutional accreditation system is new for every institution that is to be assessed and, thus, we have to maintain focus on meeting with the institutions and explaining the system s rationales, framework and processes as the institutions go into the process. Secondly, we must be aware that the new system, although simplified and less detailed, is still an external control process, and that, quite understandably, the institutions interpret our briefs, notes and letters in this context. It should also be noted, in a broader perspective, that we regularly conduct thematic analyses of important issues of quality and quality assurance. Here we aim to collect and disseminate our knowledge in a focused and accessible format. Our thematic analyses have three aims. Firstly, we hope to distribute knowledge and inspiration between the institutions of higher education by analysing key issues and by showing examples of how these issues are addressed at different HEIs. Secondly, it is our aim to inform our own practice of programme and institutional accreditation. And lastly, we hope to contribute to the public discussion on higher education with our knowledge and insight. Coordination with other authorities AI is not the only agent carrying out supervision of the quality and performance of the HEIs. Apart from accreditation, the higher education institutions also have to collaborate with the Danish Evaluation Institute (EVA) in various kinds of thematic or cross-sector evaluations. Furthermore, they are also subject to on-going ministerial supervision of key indicators such as drop-out rates and unemployment rates. Therefore, from time to time the institutions raise the question of overlapping authorities. AI understands this challenge very well, and in a coordination dialogue with both EVA and the Ministry of Higher Education, we aim to lessen the administrative burden for institutions, for example by using data collected by the Ministry in the accreditations. Although the establishment and alterations of these other supervision systems lie outside of AI s jurisdiction, 10

11 on-going coordination with the other agents in the field will be a focal point for further development. A new location in 2017 As part of a decentralisation reform, the Danish Government has decided to move 4,000 state employees from Copenhagen to other parts of Denmark. For AI this means that, in 2017, we are moving to Holbæk, a smaller city one hour s drive from Copenhagen. This could pose a challenge in keeping qualified staff members and could have negative consequences for the otherwise quite well-functioning cooperation between staff members, professionally and socially. However, AI has previous experience with major changes in the working environment. In 2013, the institution was formed on the basis of ACE Denmark and a section from EVA. This merger was prepared jointly by the management and staff, and it was executed in a cautious manner. On the basis of this recent history we remain positive regarding the move to the new location. If carried out in the same manner, it is hoped that the move will be completed without any major problems and loss of knowledge or experience within the institution. 11

12 Part 2 Accreditation and Education in Denmark 12

13 This part of the report describes the historical and legislative context of the Danish accreditation system and provides an overview of the organisational units mentioned in the report. 1.1 Historical Overview The (AI) was established by law in The establishment came as a natural extension of Denmark s adoption of the Bologna declaration in 1999, according to which Denmark is under an obligation to incorporate quality assurance in the Danish legislation on higher education. Denmark also undertook to enter into collaboration on quality assurance at European level. With the introduction of the Danish Government s Globalisation Strategy in 2006, which aimed at strengthening Denmark s position in the increasing international competition, an additional initiative was added to the quality assurance of the Danish study programmes. One of the objectives formulated in the Globalisation Strategy was for Danish higher education programmes to become world-class. In order to achieve this objective, all higher education programmes should, among other things, meet international quality standards, and the programmes should also comply with society s demand for highly qualified and competitive labour. The system was operated by two external quality assurance agencies, ACE Denmark The and the Danish Evaluation Institute (Danmarks Evalueringsinstitut (hereinafter called EVA)). Prior to the establishment of AI, the Danish Centre for Quality Assurance and Evaluation in Higher Education (Evalueringscenteret) ( ) and later EVA (from 1999) conducted a number of external programme evaluations of Danish higher education programmes. This involved a systematic evaluation of all higher education programmes according to an overall rotation plan. The results of the evaluations were not legally binding, and the evaluations could not have any financial consequences for the institutions. Between 2004 and 2007, EVA also conducted programme and institutional accreditations at mid-termed higher education level. However, the criteria were different from those after 2007 and focused on university colleges. Following experience from the first six years of programme accreditation, the Danish accreditation system was modified in The programme accreditations had shown that 85% of the programmes evaluated had achieved a positive accreditation. The HEIs raised a critique of the existing system, claiming it was unnecessarily bureaucratic and resource-intensive compared to the high number of programmes which achieved a positive accreditation. In 2011, a new government in Denmark gathered all higher education institutions under the Ministry of Higher Education and Science. Furthermore, one of the Panel s conclusions in the ENQA review of AI in 2010 was that accreditation at institutional level should be considered. AI and the Accreditation Council shared this view and contributed to the process of change prior to the new Act. The contribution included stakeholder dialogue and the Council s input to the design of the new system. On this background, the Danish accreditation system was modified by the Minister for Higher Education towards one common system for all types of higher education programme and institution, with strong focus on compliance of the HEIs internal quality assurance systems with the ESGs. As a consequence of the new Accreditation Act in 2013, ACE Denmark and the 13

14 unit responsible for accreditation in EVA were merged into one: The Danish Accreditation Institution. The first round of the new institutional accreditations was conducted during , and included eight HEIs. In January 2016, the second round of institutional accreditations is now running, supplemented by follow-up on one HEI, which achieved conditional positive accreditation in the first round. In 2016, an additional six HEIs have applied for institutional accreditation, supplemented with the reaccreditation of one HEI which achieved conditional positive accreditation in the first round. The accreditation processes will start during spring It is up to the HEIs do decide when to apply for institutional accreditation, but it is expected that all HEIs will be accredited by 2018/ The Accreditation Institution From a legislative perspective, the Accreditation Institution (AI) is anchored in the Danish Act on the Accreditation of Higher Education Institutions (Lov om akkreditering af videregående uddannelsesinstitutioner) of 2013 (hereinafter the Accreditation Act). Responsibility for the Accreditation Act lies with the Danish Minister for Higher Education and Science (hereinafter the Minister for Higher Education). AI s funding is from the annual Finance Act. In 2014, funding totalled DKK 36.6 million, with 41 employees (38,5 full-time equivalents) The Accreditation Institution s tasks The Accreditation Act sets out the responsibilities of AI, which are to ensure and document the quality and relevance of higher education programmes. AI meets these responsibilities by carrying out institutional accreditations as well as programme accreditations. Institutional accreditation is based on pre-defined criteria for the HEIs to ensure their continuing and systematic assurance and development of programme quality and relevance. Furthermore, AI carries out accreditations based on centrally defined criteria for the quality of existing and new study programmes. The accreditation tasks of AI are as follows: Accreditation of all higher education institutions in Denmark Accreditation of new and existing bachelor, master s (candidatus) and master programmes 1 at universities Accreditation of new and existing professional bachelor and academy profession programmes as well as further adult education and diploma programmes Accreditation of new and existing bachelor, master s (candidatus) and master programmes under the Ministry of Culture, including the bachelor, master s (candidatus) and master (postgraduate education) programmes in the Arts 1 In the Danish educational system a distinction is made between on the one hand master s (candidatus) degree studies, which are full time university studies principally targeting students graduating from secondary education, and on the other hand master studies, primarily offered to people in work as part-time further educational activity at university level. 14

15 Thus, following the logic of the 2013 Accreditation Act, two systems are currently in operation, but institutional accreditation is gradually succeeding programme accreditation. A clear outcome of achieving a positive institutional accreditation is that it allows the HEI the option to set up new programmes as well as justifying existing programmes. Furthermore, an HEI s existing programmes will no longer have to undergo programme accreditation. In practise, this outcome means that institutional accreditations will replace accreditation of existing programmes and provisions of programmes. Higher education programmes in Denmark are primarily anchored in the Ministry of Higher Education. A minor group of artistic education institutions are anchored in the Ministry of Culture. The accreditation criteria vary depending on the rules applying within the two ministries. As a result of the Danish system, the Accreditation Act is therefore implemented in two different ministerial orders with different sets of criteria. One ministerial order for institutions and study programmes under the Minister for Higher Education and one for institutions and study programmes under the Minister for Culture. Accreditation is mandatory and a precondition for attaining public funding for all institutions and study programmes. A very limited group of HEIs and programmes are anchored within other ministries, for instance the Police Academy and the Royal Danish Defence College. AI also conducts programme and institutional accreditations for these HEIs as revenue-funded activities. In addition to the accreditations, an important element in the Accreditation Act stipulates that AI has to gather and disseminate relevant national and international accreditation experience. AI has carried out this task by conducting annual thematic analyses. The analyses reflect the findings in the accreditation reports or other themes regarding quality assurance and educational policy issues. Furthermore, AI participates in relevant national and international conferences and disseminates new international experience and knowledge to stakeholders. In recent years, the institution has also contributed to international conferences by presenting papers. For instance at the INQAHEE conference 2015 and the EQAF AI puts a very strong emphasis on a close dialogue with all key stakeholders regarding all activities. In practice, this means that AI annually hosts start-up meetings with the HEIs prior to new rounds of programme and institutional accreditations and AI regularly meets with both the senior management and administrative units to discuss experience from previous accreditations. A recent initiative is the establishment and hosting of STAR (Student Accreditation Council), which is a discussion forum for students engaged in quality assurance and political activities at the HEIs. Additionally, the subjects for the thematic analyses are determined in dialogue with key stakeholders such as the rectors conferences for the different sectors in Danish higher education, national students associations and representatives from the Danish labour market The organisation of the Pursuant to the Accreditation Act, the Danish accreditation system formally consists of: 15

16 The Accreditation Institution, which is an independent authority within the State Administration, and which conducts accreditation of higher education institutions and their programmes The Accreditation Council, which makes decisions concerning all the above accreditation tasks The Minister for Higher Education appoints a Director nominated by the Accreditation Council, which also approves advertisements for the position. In performing her duties, the Director is not subject to the power of instruction from the Minister or the Accreditation Council. The Director has an obligation to complete accreditation reports as the basis for the Accreditation Council s decisions. In terms of budgetary matters, AI has its own budget line in the annual Finance Act and in matters of administration, the management reports to the Minister for Higher Education as the management is subject to the general rules applicable within the State Administration. As can be seen from the description of the statutory organisational units, a clear division of work has been defined between the decision-making authority, the Accreditation Council, and AI, which is also reflected in the Director s outline of the organisation. The Accreditation Council is thus an independent unit outside AI, whereas the four internal units have their separate duties. According to the Accreditation Act, AI is to serve the Accreditation Council. This is reflected in the establishment of the Council Secretariat. However, the employees of the Council Secretariat are a part of AI and contribute to projects across different areas. Management secretariat Assists the other secretariats with their work. Manages finances. Law matters. Handles internal and external communication. The Area for Professional, Vocational and Maritime Institutions (PEM) Handles accreditation of new and existing programmes and local provisions of programmes within the area. Handles accreditation of university colleges, academies of professional higher education and maritime education institutions. Takes part in Danish and international cooperation and development projects. Documents and communicates activities and results. The Area for Universities and Educational Institutions of Arts and Culture (UNIK) Handles accreditation of new and existing programmes within the area. Handles accreditation of universities and art academies. Takes part in Danish and international cooperation and development projects. Documents and communicates activities and results. The Council Secretariat Provides the Accreditation Council with service. 16

17 Prepares Council meetings. Is responsible for the Council s communication with AI. Maintains contact with the Council s stakeholders. The two areas will be named PEM and UNIK in the following. Figure 1: Organisational chart of the Accreditation Institution 17

18 1.3 The Accreditation Council Accreditation Council Responsibilities and Organisation According to the Accreditation Act, the Accreditation Council is the specific body which makes the decisions regarding accreditation of all higher education institutions and their programmes based on accreditation reports prepared by the. The Council has authority to award, conditionally award, or deny accreditation for all higher education institutions and their programmes. The Council s decisions are based on the overall assessment and recommendations of the accreditation reports. The Accreditation Council holds approximately five meetings a year. At its meetings, the Council makes decisions regarding accreditation of higher education institutions, study programmes and the provision of programmes, in addition to considering a number of different issues of relevance to the accreditation task. The Accreditation Council defines its procedures and methods independently of both political and other institutional interests. The Council has overall responsibility for ensuring the quality and relevance of higher education programmes. The Council s decisions and the basis for these decisions are accessible to the public, as both the decisions and the accreditation reports are published on In addition to the Accreditation Council meetings, the Council Chair and Vice-chair (the Chairmanship) meet regularly with AI to plan the Council s meetings and deal with procedural problems and issues of principle as may arise in the course of their accreditation activities. Moreover, the Council meets with external stakeholders such as the Minister of Higher Education and the stakeholder organisations of the educational institutions in order to discuss issues of principle. Composition of the Accreditation Council The Accreditation Council has a Chairman, a Vice-chairman and seven other members, including two student members. The Minister for Higher Education appoints the Chairman. The rest of the Council s members are appointed by the Minister for Higher Education on the basis of recommendations from relevant institutions, organizations etc. The members of the Accreditation Council are appointed based upon their experience and knowledge of quality assurance, higher education, research and development and labour market conditions. At least one of the members must have international accreditation experience. The Council must be composed of an equal number of men and women. The Chairman and the members are appointed for a period of four years. However, the two student members are only appointed for a period of one year. Members are eligible for reappointment once. The As can be seen from section 1.2, the is the operator for all higher education institutions and study programmes in Denmark. The Danish Accreditation Institution also collects national and international experience of relevance to accreditation. The 18

19 s performance of its tasks and responsibilities are described in further detail in Part 2. Other operators According to the Accreditation Act, at its own initiative or following a request from a university, the Accreditation Council may base its accreditation decision fully or partly on an accreditation report from another internationally recognised institution, to the extent that such report is prepared in accordance with the same criteria as other accreditations within the field in question. So far, this option has never been exercised. The Advisory Committee to Assess the Range of Higher Study Programmes Offered (RUVU) In 2013, the Minister for Higher Education established the Advisory Committee to Assess the Range of Higher Study Programmes Offered (hereafter RUVU). This is a committee dedicated to deciding on the relevance of proposals for new programmes. RUVU is a part of the overall accreditation architecture, but is an independent committee within the ambit of the Ministry of Higher Education, and with no organizational affiliation to the. RUVU issues a recommendation of prequalification based on an assessment of the relevance or pertinence of the proposed new study programme. It is the Minister of Higher Education who on this basis formally decides on whether a programme is to be prequalified, and thus allowed to apply for accreditation. This decision is taken prior to any steps being taken to initiate the Accreditation Institution s accreditation processes and the Accreditation Council s decisions. In practice, the establishment of RUVU has meant that assessment of the relevance of new programmes is removed from the accreditation process. 1.4 The Accreditation Council as Decision Maker The Accreditation Council decides on accreditation of all higher education institutions and their programmes in Denmark, regardless of the ministry under which such programmes belong. The Accreditation Council fulfils this role by, among other things, entering into a broad dialogue with the most important stakeholders. Thus, the Chairmanship in particular engages from time to time in dialogue with the Minister for Higher Education, Parliamentary spokesmen on higher education policy, and with the various rectors conferences/rector chairmanships. Moreover, the Council hosts conferences that bring together many different players within the field of higher education in order to consider questions relating to quality assurance. Finally, it should be stressed that, due to its decision-making authority, the Accreditation Council is subject to public attention in connection with the announcement of its decisions and communication of the consequences of its decisions for the accredited programmes Decision-making Process of the Accreditation Council The Accreditation Council makes independent decisions based on accreditation reports prepared by the accreditation operators. As can be seen from part 3, the reports contain recommendations which form the basis of the Council s decisions. However, the Council is not 19

20 bound by these recommendations and, if it sees reason to do so, may decide not to follow a recommendation. The present Accreditation Council was appointed in January 2014 and has enjoyed considerable stability among its members since. One of the present members was also a member of the former Council, and this overlap ensures consistency between the present Council and the former Council s experience. Even though this has created a sound basis for ensuring coherence and continuity in the Council s decisions, the Council is also working continuously to optimize its decision-making process, including the preparations which are necessary to ensure efficient and credible decisions based on a large number of accreditation reports. Finally, it should be mentioned that the always attends the Accreditation Council meetings so as to answer any questions from Council members. The Council members are requested to send any questions to AI a few days beforehand in order to allow the operator time to prepare. It is important for the Accreditation Council to be well acquainted with the procedures and methods which form the basis of the decision-making platform presented to the Council. Therefore, the Council is presented to the operators procedures and methods on a continuing basis. The procedures and methods are regulated under the auspices of the Danish Public Administrations Act, which governs instructions concerning the legal position of citizens towards the civil administration. Thus, the Council s feedback to the Danish Accreditation Institution is focused on enhancement and alignment in order to ensure transparency and equality. 1.5 Outline of the Danish Higher Education System This section provides an overview of the Danish higher education programmes. See also Annex H: Framework for Higher Education Institutions in Denmark Higher Education Institutions and Study Programmes The higher education system under the Ministry of Higher Education in Denmark includes eight universities, eight university colleges, nine academies of professional higher education, three artistic education institutions and 10 maritime education institutions. In total, there are approximately 240,000 students at higher education institutions under the Ministry of Higher Education. The number of applications for higher education programmes has grown in recent years. All higher education institutions are primarily publicly funded and are subject to various types of regulations and state supervision. In addition, there are seven HEIs under the Ministry of Culture and a few institutions under the Ministry of Defence and the Ministry of Justice. Accreditation is mandatory for all higher education under the auspices of the Ministry of Higher Education and the Ministry of Culture. 20

21 Figure 2: Overview of the Danish higher education institutions under the Ministry of Higher Education Danish higher education programmes are divided between research-based and professionally based programmes. The purpose of the research-based programmes is to educate students to the highest international standards within and across the research-based disciplines, whereas the purpose of the professionally oriented programmes is to ensure education closely based on practice, and at a national level to meet the need for well-qualified professionals in the private and public sectors. The and the Accreditation Council cover all types of higher education institutions and study programmes in Denmark. In Denmark, it has been decided politically that PhD programmes are not subject to accreditation. Research-based programmes are offered by the universities, and regulated by the Ministry of Higher Education in the Danish University Act (Universitetsloven) (Annex xx). There are approximately 1,050 research-based study programmes, educating approximately 140,000 students. The professionally oriented programmes, with approximately 100,000 students, are 21

22 predominantly offered by the University Colleges and the Academies of Professional Higher Education. Whereas the university programmes are research-based, these programmes are based on up-to-date knowledge, recent research results and close contact with practice. These programmes and institutions are also regulated by the Ministry of Higher Education. The programmes fall under the Danish Act on Academy Profession and Professional Bachelor Programmes (Lov om erhvervsakademi- og professionsbacheloruddannelser). Furthermore, a small group of maritime education programmes are offered by the Maritime Education Institutions. Finally, there are the professionally oriented programmes offered at institutions under the auspices of other ministries, e.g. the Ministry of Defence and the Ministry of Justice. Most Danish institutions of higher education have been merged over the last years to form larger units. This means that the institutions have been strengthened with more students and a broader provision of study programmes, but also that the administrative systems have undergone adaptation and development Qualification Levels in the Danish Higher Education System The Danish higher education system is organised into four qualification levels, with a number of both ordinary and adult further education degree types at each level. This is illustrated in Figure 3 below. The Danish qualification framework systematically describes the different degree types within the Danish higher education system. Figure 3: Qualification levels in the Danish higher education system 22

23 Part 3 Compliance by the Danish Accreditation Institution with the European Standards and Guidelines for the External Quality Assurance of Higher Education 23

24 ESG Part 2 - Compliance by the with the ESG for External Quality Assurance ESG 2.1 Consideration of internal quality assurance STANDARD: External quality assurance should address the effectiveness of the internal quality assurance processes described in Part 1 of the ESG. GUIDELINES: Quality assurance in higher education is based on the institutions responsibility for the quality of their programmes and other provision; therefore it is important that external quality assurance recognises and supports institutional responsibility for quality assurance. To ensure the link between internal and external quality assurance, external quality assurance includes consideration of the standards of Part 1. These may be addressed differently, depending on the type of external quality assurance. Later in this report, under ESG 2.5, the criteria that the institutions and study programmes must comply with in institutional accreditations and programme accreditations are described in detail. The criteria were established by the Ministry of Higher Education. Thus, the Danish Accreditation Institution is responsible for implementation of the criteria, but responsibility for any alterations to the criteria lies with the Ministry. In general, institutional accreditation covers the following areas in ESG Part 1: ESG 1.1: policy for quality assurance (Criterion 1) ESG 1.2: design and approval of new study programmes (Criterion 5) ESG 1.3: student-centred learning (Criterion 4) ESG 1.4: student admission and progression (Criteria 4) ESG 1.5: teaching staff related to the programmes (Criterion 3) ESG 1.6: facilities and learning resources (Criterion 4) ESG 1.7, ESG 1.9 and ESG 1.10: cyclical external quality assurance as well as analysis and use of information for effective management and on-going monitoring of programmes (Criteria 2, 3, 4 and 5) ESG 1.9: contact and dialogue with labour market stakeholders for the monitoring of the programmes (Criterion 5) In general, programme accreditation covers the following areas in ESG Part 1: ESG 1.3: student-centred learning (Criterion 4) ESG 1.4: student admission and progression (Criteria 3 and 4) ESG 1.5: teaching staff related to the programme (Criterion 2) ESG 1.6: facilities and learning resources (Criterion 5) ESG 1.7 and ESG 1.9: analysis and use of information for effective management and ongoing monitoring of programmes (Criterion 5) 24

25 ESG 1.9: contact and dialogue with labour market stakeholders for the monitoring of the programmes (Criterion 1) Thus, the considers that it meets the part 1 standards to the extent that institutional accreditation reviews the effectiveness of the HEIs internal QA processes, and that the accreditation process for programme accreditation reviews the HEIs quality assurance systems for the programme as one of five criteria. A key element in both types of accreditations is that responsibility for the quality assurance of study programmes and institutions lie with the HEI itself. Accreditation is one important external element aimed at ensuring the quality of the HEIs study programmes. The legislation on higher education is another element, which also represents an important framework for the organisation and quality assurance of study programmes. Among other things, the Danish legislation ensures that student assessments are based on published criteria, rules and procedures. These are set out in the following ministerial orders: the Danish Ministerial Order on Bachelor and Master's (Candidatus) Programmes at Universities ( Uddannelsesbekendtgørelsen ), the Danish Ministerial Order on Academy Profession Programmes and Professional Bachelor Programmes ( Bekendtgørelse om erhvervsakademiuddannelser og professionsbacheloruddannelser ), the Danish Ministerial Order on the Grading Scale and Other Forms of Assessment of Institution Education ( Karakterbekendtgørelsen ) and the Danish Ministerial Order on Examinations ( Eksamensbekendtgørelsen ). The Danish Ministerial Order on Examinations also ensures that a minimum of 1/3 of the exams in every study programme are carried out with the participation of an external examiner, whose main task is to ensure the fairness and objectivity of the examination of the students. The organisation and the activities at the HEI s are in general regulated by a number of Acts and Ministerial Orders. As a consequence, not all aspects mentioned in ESG Part 1 are subject to accreditation, because these activities are already regulated by the national legislation and/or labour market agreements. Aspects which are not subject to accreditation are: Number of students admitted (part of ESG 1.4), recruitment of teaching staff (part of ESG 1.5), funding per student FTE (part of ESG 1.6), public information (ESG 1.8). In recent years, the Danish HEI s have endeavoured to establish and systematise their work on internal quality assurance systems. At present, institutions differ as to how far they have come in this work. Supporting the development of institutions quality assurance systems has therefore constituted a key focal point for the guidelines for the accreditation of institutions which came in effect in July 2013 (see also ESG 2.2). The institutional accreditations that have already have been decided on by the Council reflect the more diverse and developmental approach to external quality assurance. Firstly, very different systems and institutions, from the Technical University of Denmark to University College Sjælland, have received a positive accreditation. Secondly, institutional accreditation is an external quality assurance system with high threshold levels, and just four of the first nine 25

26 institutions have received a positive accreditation. Four institutions have received a conditional accreditation and one has received a refusal of accreditation (as of January 2016). As mentioned earlier, the Danish system of quality assurance of higher education is based on a political decision to shift the focus from programme accreditations to institution accreditations. However, in a transition phase, until all institutions are accredited, AI must make sure that also the non-accredited institutions are ensuring the quality of their programmes. As a consequence, programme accreditation is still carried out in these institutions, focusing on the larger programmes with a high number of students, or on faculties and institutions that have received a relatively higher number of less than positive accreditations in the past. 26

27 ESG 2.2 Designing methodologies fit for purpose STANDARD: External quality assurance should be defined and designed specifically to ensure its fitness to achieve the aims and objectives set for it, while taking into account relevant regulations. Stakeholders should be involved in its design and continuous improvement. GUIDELINES: In order to ensure effectiveness and objectivity it is vital for external quality assurance to have clear aims agreed by stakeholders. The aims, objectives and implementation of the processes will bear in mind the level of workload and cost that they will place on institutions; take into account the need to support institutions to improve quality; allow institutions to demonstrate this improvement; result in clear information on the outcomes and the follow-up. The system for external quality assurance might operate in a more flexible way if institutions are able to demonstrate the effectiveness of their own internal quality assurance. AI has continually developed and amended the accreditation concept with the intention of ensuring an efficient accreditation concept in the sense that the criteria are transparent and predictable and at the same time wide enough to encompass very different types of programmes and institutions. It is important for AI that key stakeholders are involved in this work: - The institutions and their common representation, such as Danish Universities, University Colleges Denmark and Danish Business Academies - National student organisations - The Danish Ministry of Higher Education - Industry/labour market representatives, such as the Confederation of Danish Industries and the Confederation of Danish Employers - Trade unions The following section provides a description of the shift towards institutional accreditation and how the stakeholders are involved in development of the accreditation concept and guidelines. Furthermore, AI s procedure for the publication for guidelines and accreditation reports is outlined in section ESG 2.6. Regarding solutions for specific quality issues, the institutions can demonstrate their improvement in various ways; most notably in the follow-up process after a conditional accreditation (see section ESG 2.3). Fit-for-purpose an overarching principle in accreditation In 2013, when the accreditation system was amended, one of the main objectives was to establish a system that was both fit-for-purpose and also ensured less work load for the institutions involved. One way to lessen the workload and the cost for the institutions when being accredited is to ensure their involvement in the drafting of the guidelines that interpret the criteria. This ensures transparent criteria for assessment and a mutual understanding of the 27

28 needs for documentation. Another way is to make sure that the institutions can extensively use pre-existing documentation to show that they meet the criteria, such as annual reports to the board on study programmes, and minutes or key indicators that are already collected because they inform about ongoing quality assurance work. Involving the institutions and lessening their workload are conditions that AI finds important and takes into account when developing methodologies and conducting accreditations. Programme accreditation is largely comprised of a fixed set of assessment points that the programme must accommodate. With institutional accreditation the institutions themselves are responsible for designing and applying a system that ensures and develops the quality and relevance of their programmes. The institutions have the freedom to design a quality assurance system that reflects their programmes and any special circumstances that might pertain to them. Hence, in institutional accreditation, there is stronger focus on policies and strategies affecting quality assurance, and holistic assessments are made that take into account whether the institution s quality assurance policy and procedures are fit-for-purpose. With the introduction of institutional accreditation, the control perspective on quality assurance is now combined with a more developmental perspective. As part of the self-evaluation report, the institutions are to reflect upon the strengths, weaknesses and potential for improvement within their own quality assurance systems, and take into consideration, for instance, the way in which the system functions in relation to the specific character of the institution, its visions, objectives and challenges. Because the higher education sector is composed of institutions that have different knowledge bases, size and institutional set ups, an important aspect of institutional accreditation is that it allows for different quality assurance systems, provided the dispositions within them are wellsubstantiated and of course meet the criteria for quality and relevance laid down in the Ministerial Order. It is thus stated in the guidelines that: Institutional accreditation places the responsibility for the quality of programmes squarely with the institution and the institution management. This means that the institution must have established a quality assurance system that reflects the programmes at the institution. (Annex D: The Guide for institutional accreditation, p. 3). When drafting the criteria, it was important to AI, that the criteria took into account the heterogeneity of the HEIs. There is focus on the overall framework for quality assurance at the institution, and the criteria concerning how it is put into practice are also phrased in inclusive and broad terms to take into account the heterogeneity of the HEIs (see ESG 2.5). At the same time, this consideration must be balanced with the need for a transparent system where it is clear to the institutions that they are assessed on equal terms. The choice of using audit trails is another way to ensure methodologies are fit-for-purpose. This method ensures that the accreditation panels have an opportunity to select several areas of interest that they can examine more closely. This can be areas of general importance, but most often it will be areas of interest due to the specific characteristic of the institution being accredited. The institutions are consulted in identifying the relevant interviewees and the 28

29 relevant documentation, and they are also given the opportunity to suggest the general themes for audit trails. Stakeholder involvement in the development of methodologies used in institutional accreditation In the day-to-day operations and the collaboration with the HEIs, a number of activities are undertaken to ensure a good dialogue and mutual adjustments of expectations. Stakeholders are deeply involved on an ongoing basis in the development, adjustment and adaptation of the accreditation concept and guidelines; and most often in the form of a written hearing. This gives allows the institutions to provide input and to point out any errors, omissions or inadequacies in the guidelines, which AI can then rectify. With the transition to institutional accreditation, a new set of guidelines also had to be drafted. They had to cover several new topics and the involvement of relevant stakeholders in this process was more extensive. AI held different types of seminars to ensure a thorough and transparent process incorporating the stakeholders input into the final guidelines and methodologies. The Accreditation Council had its main focus on the overall policy level of the amended system, and invited stakeholders to two conferences, whereas the main focus of AI was the operational level. Since AI provides general secretariat assistance to the Council, the different activities were complementary, and there was a high degree of knowledge sharing between the Council and AI. In 2012, one year prior to the adoption of the amended system, the Accreditation Council held a conference on quality assurance in higher education and perspectives for the upcoming system. The invitation to participate was extended to the entire educational sector, and the Minister for Higher Education opened the conference by highlighting the main expectations for an amended system. Later that year, a seminar aimed at a narrower group of participants, such as board members at the institutions, rectors and political spokespersons, was held by the Council. The seminar focused on quality culture and management responsibility for quality assurance within an amended system. The management of AI participated in both seminars In spring 2013, AI held two dialogue meetings. One was aimed at labour market representatives, the other at the institutions. Prior to these meetings a draft of the concept was provided, which, for instance, enabled AI to gather input on how to organise and optimise the use of audit trails. After these meetings, the concept was revised. When the guidelines were finalised by AI, a formal hearing at the institutions was conducted. An overview of similar international experience was also established and taken into account. In 2014, after the completion of the first round of institutional accreditations had been completed, an evaluation meeting was held with each of the accredited institutions. The topics for these meetings were the process itself and to what extent the application of the guidelines had been fit-for-purpose for the specific institution. Every institutional accreditation will be finalised with such an evaluation meeting to ensure the ongoing adjustment of methodologies (Annex XXX). 29

30 Overall, there has been positive feedback from these evaluation meetings regarding the level of detail in the reports and the way audit trails have been applied and organised, for instance. However, the evaluation meetings also increased the awareness of a continuous need for AI to be very clear and specific in relation to the type and scope of documentation necessary for each accreditation. The documentation requirements were characterised as quite burdensome by all the institutions, and it is clear from the evaluation meetings that AI has yet to find the optimal balance between having sufficient material for the accreditation panel to make sound, fair and nuanced assessments and not requiring too much documentation from the institutions. Besides broader criteria, new elements have also been introduced, and AI has produced several memoranda along with the guidelines: a memorandum that provides a preliminary understanding on what AI expects of institutions applying for institutional accreditation, a memorandum on what constitutes tested practice and a working paper on quality culture. These memoranda are produced with the intention of securing a transparent concept in which it is clear to the institutions what is expected of them. Stakeholder involvement in the ongoing development and adaptation of methodologies The latest more comprehensive adjustment of the guidelines for programme accreditation took place in 2013 as a consequence of the new Accreditation Act. A number of different guidelines aimed at the research-based programmes on the one hand, and the professionally based programmes on the other hand, were gathered in a set of common guidelines for the HEIs. The intention was to simplify the guidelines and secure their applicability at all kinds of higher education programmes. The revised guidelines were subject to a formal hearing and following adjustment. In institutional accreditation, broad criteria are drafted to accommodate quite different types of institutions. Broader criteria mean that there is an increased need for information and ongoing adaptation of expectations as well as a need for guidance and support of the institutions in interpreting and translating the criteria to the specific institution. As a consequence of this, an information meeting is held with the institutions prior to the onset of every accreditation process. At this meeting, the guidelines are explained and the institutions are given the possibility to clarify any questions they might have regarding the process itself or more specific elements in the guidelines and assessment criteria. They are also given the opportunity to make AI aware of any special circumstances pertaining to their programme or institution. The institutions are also given the opportunity to point at relevant academic areas and labour markets which AI can take into consideration when an accreditation panel is appointed. When the accreditation panel has been appointed, the institutions are given the possibility to comment on the composition of the accreditation panel. This is to ensure not only the panel members competence to act, but also that the right professional skills are represented in the panel. 30

31 In general, AI has an ongoing dialogue with the institutions throughout every accreditation process to address any need for further clarification or additional questions that might arise. Every accreditation process includes multiple exchanges of written information, where the institutions are given the opportunity to explain specific circumstances to the panel and AI. Student involvement Student representatives have been included in the accreditation panels from the very establishment of the system in With the transition to institutional accreditation, AI saw a need for further formalisation of the ongoing dialogue with the student organisations on issues related to quality assurance and to gather their views on the usefulness of the methodology applied in the accreditation work. For that purpose, STAR (the Students Accreditation Council) was established in Nineteen national student organisations are represented in STAR. The organisations appoint their representative themselves. STAR is used as a sounding board and makes no formal decisions related to accreditations. STAR meets approximately twice a year and, amongst other things, it has discussed the transition from programme to institutional accreditation and the students role in the institutional accreditation meetings. Motivated by a concern voiced by STAR as to whether the students participating in the meetings in the first round of institutional accreditation were representative of the entire institution, AI drafted a paper that gave an overview of the students who had participated in the first round of institutional accreditation meetings and the criteria used in their selection. This discussion has made AI aware of the importance of explaining the process to both the students in general and specifically to the students involved in the process. To more broadly inform students who are interviewed (or in other ways involved in accreditation), the students now receive an information letter prior to the site visits, explaining the concept and the process. Thematic analyses Every year, AI conducts a number of thematic analyses (see also ESG 3.4). Over the past couple of years, external stakeholders such as the rectors conferences within the different sectors, representatives from the labour market and STAR have been invited to discuss the proposed subjects for the analyses prior to their start-up. This has given AI an indication of the relevance of the proposed analyses to our stakeholders, and input regarding possible elements to take into consideration when conducting the analyses. Furthermore, it has given AI input for possible upcoming analyses. The external stakeholders are also involved in the thematic analyses through their participation in surveys, interviews etc. The themes for the analyses are also inspired by current topics and developments within the educational field as well as challenges at the institutions that are identified in the accreditation processes. One example is the publication on interdisciplinary study programmes conducted in The analysis was initiated, because AI noted a rise in applications for such programmes, and therefore saw a need to identify possible quality challenges specific to these programmes and methodological challenges when applying the criteria. Another example is the analysis 31

32 carried out in 2015 regarding challenges and strengths with small and multiple campuses. This has been, and will be, an issue in several institutional accreditations, and the conclusions from the thematic analyses will therefore enlighten the analyses carried out in both ongoing and future programme and institutional accreditations. Hence, the thematic analyses conducted are inspired by the accreditations, and they can also enlighten the methodologies used by AI and contribute to ensuring that the assessment criteria can encompass a wide range of quality issues across the higher education sector. Systematic gathering of users viewpoints Feedback regarding programme accreditation is collected as a part of the systematic evaluations of user opinions and experience with both the guidelines and the accreditation process which AI has carried out since These evaluations have mainly been carried out as electronic surveys. The surveys have included institutions which have had programmes accredited. Members of the accreditation panels used in both programme and institutional accreditation have also been surveyed. The latest survey amongst and panel members was carried out in autumn 2015, and a survey amongst institutions is to be completed in the beginning of AI has not yet (as of December 2015) decided on any adjustments on the basis of the latest survey. In conclusion, the formal dialogue with stakeholders, seminars, evaluations and surveys are complemented by an ongoing and also very valuable informal dialogue with the institutions. All sources are used to adjust, improve and optimize AI methodologies and support the need the institutions might have to improve quality. At the same time AI is aware that the involvement of stakeholders must not be at the expense of the independence of the institution. 32

33 ESG 2.3 Implementing processes STANDARD: External quality assurance processes should be reliable, useful, pre-defined, implemented consistently and published. They include a self-assessment or equivalent; an external assessment normally including a site visit; a report resulting from the external assessment; a consistent follow-up. GUIDELINES: External quality assurance carried out professionally, consistently and transparently ensures its acceptance and impact. Depending on the design of the external quality assurance system, the institution provides the basis for the external quality assurance through a self-assessment or by collecting other material including supporting evidence. The written documentation is normally complemented by interviews with stakeholders during a site visit. The findings of the assessment are summarised in a report (cf. Standard 2.5) written by a group of external experts (cf. Standard 2.4). External quality assurance does not end with the report by the experts. The report provides clear guidance for institutional action. Agencies have a consistent follow-up process for considering the action taken by the institution. The nature of the follow-up will depend on the design of the external quality assurance. AI has a systematic practice to ensure that study programmes are assessed in a consistent manner, including a self-assessment report, one or two site visits, an accreditation report and follow-up-procedures. In addition, the processes and criteria of the Accreditation Council and AI are known and published prior to the accreditations, as stated in the last section in ESG 2.2. The rotation plan for both institutional and programme accreditation is established with the participation of the HEIs. The institutions decide themselves, when they are ready for institutional accreditation, and programme accreditation plan is established in a process including a formal hearing of the institutions. Institutions that are in the process of an institutional accreditation are not programme accredited. The entire accreditation process is described on the website and in the guidelines. The website also has guidelines for HEIs at the start-up of each accreditation round, including a step-by-step description of the accreditation process. In addition, AI holds a start-up meeting at the institutions with study programmes that are included in the upcoming accreditation round. 33

34 Self-assessment report Every accreditation process at both institution and programme level includes a self-assessment report, where the institution is given an opportunity to present the programme/institution at its own terms within the criteria framework. In this self-assessment, the institution can present the context-specific dilemmas and challenges, which they deem relevant for the accreditation panel and AI to take into account in their assessment. Site visits Accreditations of existing programmes and institutions include a site visit, but accreditations of new programmes do not. This is described further in the following. In an existing programme accreditation, the visit to the institution process lasts one day and is organised such that the accreditation panel and AI meet the management, lecturers and students. For historical reasons, there is a small difference between site visits in PEM and in UNIK. PEM asks the institutions to invite the students whose birth date is closest to a specific month, and this ensures representativeness by random selection. UNIK asks the institution itself to ensure that the students are representative. Because the PEM HEI sector is closer linked to the labour market, the accreditation panel will also meet with coordinators of the student traineeships, if deemed relevant at the specific programme. Furthermore, AI requires that the different groups meet separately so that there are no repeat participants from one meeting to another. AI considers it very important that the staff can express themselves freely without the presence of management, and in the same way it is important that students can express themselves freely without the presence of management or staff. The presence of management or staff could have the unwanted effect that students express themselves more cautiously. In addition, an overlap of participants between meetings might mean that the responses do not cover quite as many different angles. The day begins and ends with a meeting with the management. Key issues discussed in the course of the day are reviewed at the final meeting with the management to ensure that the management is heard on all key issues (Annex G: Procedure Handbook). There is no site visit in the process for accreditation of new programmes. The purpose of a site visit is to meet with the management, the teachers and the students to get an impression of a study programme in practice. However, new study programmes are only implemented upon approval by the Council. Thus, there is no practice to examine and assess in the accreditation process. This means that the panel s assessments in accreditation of a new study programme are entirely based on the institutions written documentation. In all types of accreditation, the accreditation panel and AI can request further explanations from the institution, if the written documentation is not sufficient. In an institutional accreditation there are two visits to the institution; a shorter first visit and a slightly longer second visit. The purpose of the first, introductory meeting is to provide the panel members with insight into the institution's overall quality assurance policies and systems (focus on criteria I-II) and to find a basis for selecting the audit trails that will be the object of the second visit. The first visit includes meetings with the institution's top management, including 34

35 representatives of the board of governors, representatives of other management levels, teachers, students, representatives of the potential employers' panels and, if deemed helpful, also administrative personnel. The accreditation panel makes a decision about the audit trails that will be used to illustrate the performance of quality assurance in practice. The first visit lasts two days. Using audit trails, the purpose of the second visit is to assess how the performance of quality assurance functions in practice (focus on criteria III-V). The visit includes meetings with management at various organisational levels, teachers, students and other members of staff that might have relevant information concerning the chosen audit trails. The duration of the second visit varies according to the size of the institution, including its geographical distribution and the need for audit trails. The second visit lasts from two to five days. Accreditation reports Every accreditation process leads to an accreditation report. AI has established the following processes in respect of ensuring consistency in the reporting: Preparation of the accreditation report Expert accreditation panels are used in connection with the accreditation of existing study programmes in order to ensure a uniform assessment of closely related programmes from an academic point of view. On a training day, the accreditation panels are instructed in the criteria and are made aware of the importance of consistency in the criteria assessments (see section ESG 2.4 for details on the training day). The accreditation panels carry out an assessment of whether the study programme meets the individual criteria, and these assessments form the basis of AI s recommendation for the Accreditation Council. The Council makes the decision regarding accreditation. Internal feedback to all parties Two experienced accreditation consultants always provide feedback to the individual consultant on each accreditation report. The feedback process is anchored in a feedback group. The Director of Operations is a member of the feedback group and is responsible for ensuring consistency in the application of the assessment principles. The directors of operations are involved in issues of principle on an ongoing basis. Moreover, the Executive Director and the relevant Director of Operations hold a meeting at which matters of principle are discussed. They also discuss accreditation reports which recommend conditionally positive accreditation or refusal of accreditation. Matters of principle are also discussed in regular meetings with all staff from PEM and UNIK present (see also ESG 3.6) Accreditation reports submitted for hearing Prior to being considered by the Accreditation Council, the accreditation reports are submitted for a formal hearing at the HEI. The hearing aims to ensure that there are no factual errors in the accreditation report. The hearing response in respect of the study programme may give rise to adjustments in the accreditation report, the criteria assessments and the recommendation. 35

36 Decisions and follow-up procedures The Accreditation Council and AI have clear follow-up procedures in place in relation to conditional positive accreditations and refusal of accreditations. These are explained in the following outline of the possible decision categories in an institutional accreditation as well as in a programme accreditation. Positive accreditation For study programmes and institutions with a positive accreditation, follow-up by AI is only carried out once the study programme/institution again forms part of the rotation cycle. These study programmes may in addition to the decision on the positive accreditation receive recommendations from the Accreditation Council in the form of minor critical factors and comments. The HEIs will always receive recommendations from the accreditation panel in addition to a positive institutional accreditation. Recommendations or minor critical factors have no bearing on the formal decision. It is up to the institutions alone to act on the minor critical factors. Follow-up to a positive institutional accreditation will be carried out in the next six-year cycle, and AI will prepare the institutions for this process in due course. Conditional positive accreditation Institutions or study programmes with a conditional positive accreditation do not meet one or more accreditation criteria. AI s decision contains information on the following: The reason why the study programme/institution has received a conditional positive accreditation In which areas follow-up must be carried out in order to achieve a positive accreditation for the remainder of the six-year accreditation period When the study programme/institution needs to go through a follow-up process with AI based on a supplementary accreditation report. The duration of a conditional positive accreditation is two years or less. Within a maximum of two years, the Accreditation Council must have made a follow-up decision. During this time, the institution/study programme has the possibility to implement changes and improvements in the areas that were determining for the conditional positive accreditation. Usually, only the criteria stated by the Accreditation Council to be problematic are subject to assessment in the subsequent accreditation process. However, if major changes covering other criteria have been made at the institution/programme to solve the problems, these criteria can also be subject to assessment. A key point in a follow-up process is that the institutions are to find their own solutions to the problems pointed out by the panel and the Council. The responsibility lies with the institutions, since they must have the autonomy to decide for themselves which alterations and developments are deemed fit for the specific programme or problem. Based on a subsequent accreditation report, the Council can then either give a positive accreditation or a refusal of accreditation. 36

37 New study programmes cannot receive a conditional positive accreditation, only a positive accreditation or refusal of accreditation. Refusal of accreditation Study programmes which receive a refusal of accreditation cannot be approved and thus lose the right to public financial subsidies and the institution loses the right to award a degree for the programme. The Minister for Higher Education must subsequently lay down a plan for how students enrolled on the study programme can complete a programme in a manner which is most expedient for the students. Institutions which receive a refusal of institution accreditation have to return to programme accreditation of their study programmes. The institution will, in dialogue with the Council, determine when it can re-apply for institutional accreditation. Following a refusal of institution accreditation, the institution cannot establish new programmes until a new decision of either conditional positive or positive accreditation has been awarded by the Accreditation Council. 37

38 ESG 2.4 Peer-review experts STANDARD: External quality assurance should be carried out by groups of external experts that include (a) student member(s). GUIDELINES: At the core of external quality assurance is the wide range of expertise provided by peer experts, who contribute to the work of the agency through input from various perspectives, including those of institutions, academics, students and employers/professional practitioners. In order to ensure the value and consistency of the work of the experts, they are carefully selected; have appropriate skills and are competent to perform their task; are supported by appropriate training and/or briefing. The agency ensures the independence of the experts by implementing a mechanism of noconflict-of-interest. The involvement of international experts in external quality assurance, for example as members of peer panels, is desirable as it adds a further dimension to the development and implementation of processes The accreditation concept has been organised in a manner that it is fit for its defined and published purpose. AI has adopted principles for the recruitment of members for the accreditation panels which ensure international representation in the panels as well as participation by a student and an employer. AI holds a training day for the panel members, and a clear concept for site visits has furthermore been laid down. There is also a schedule for the overall accreditation process. Accreditation process Prior to each accreditation round, AI holds a start-up meeting with the institutions to be accredited (see also ESG 2.2). The meeting is used to present and discuss the process and criteria. AI sets up the accreditation panel prior to or during the time when the institutions write up their self-assessment report. The accreditation panel for institutional accreditations is composed in such a way that it covers the relevant knowledge about, and experience with, quality assurance of a similar institution, management of a similar institution, and quality assurance from another sector than the higher education sector. The panel includes a student with board experience from a similar institution or with other management experience of an HEI. 38

39 The panel for existing programme accreditations is composed in such a way that it covers the disciplines and subject areas of the study programme from an academic perspective. The accreditation panel includes a student, an employer representative and two experts within the subject. For study programmes at the universities, the experts within the subject are academics conducting research within the same subject area at an institution outside of Denmark. The panel for accreditations of new programmes also consists of a student and two experts within the subject area, but there is no employer representative in the panel for the accreditations of a new study programme. This panel composition is chosen by AI, since programmes to be accredited are only those that have already been approved by the RUVU (cf. Chapter 1.3), which decides whether there is a need for the programme on the labour market. Through their professional experience, all experts must fulfil the prerequisites for being able to assess the study programmes/institutions. To ensure this, a number of specific requirements have been laid down for the individual types of expert on which the selection is based (Annex G: Procedure Handbook). Usually, each programme accreditation panel consists of four members. The panel for an institution accreditation consists of three to six members, including the Chair of the Panel. The members of the accreditation panel in all types of accreditation participate in a training session. The purpose of this is to enable the panel members to carry out assessments of the quality of higher education within AI s accreditation concept. They thus obtain: Knowledge about accreditation in Denmark and about the Danish educational system. Knowledge of and an insight into the accreditation process and the accreditation criteria. The training session provides the panel members with a clear understanding of their role, responsibility and the scope for assessment. Great emphasis is placed on the national context (the Danish education legislation that the institutions must comply with), the accreditation methodology, and how the various criteria are to be assessed in a consistent manner. The guidelines and the criteria are reviewed, and cases from previous accreditations are presented. The self-assessment report (usually pages with annexes covering significantly more pages) is received electronically. AI distributes the report to the accreditation panel. Reading guidelines and a work plan have been drawn up for use in connection with the panel s preparations. Prior to each site visit, AI and the panel also hold a preliminary meeting. At the preliminary meeting, the accreditation panel and AI discuss the self-assessment report and, together, identify the criteria which have not been sufficiently clarified, in order to establish a solid foundation for assessment. The deliberations are gathered in draft questions that form the basis of the panel s dialogue with the institution during the site visit. 39

40 See ESG 2.3 for a description of the site-visits that are carried out in accreditations of existing programmes as well as in institutional accreditations. After the visit to the institution, the panel carries out an assessment of the study programme/institution in relation to each individual criterion. Staff members from AI write a summary of assessments and reasons in an accreditation report, which also contains an overall recommendation on the study programme. The panel receives the report for review and approval in order to ensure that it correctly and sufficiently reflects the panel s assessments and reasons. The submits an accreditation report for written consultation and comments from the HEI. The institution has the opportunity to make corrections and comments on factual conditions as well as point out deficiencies, if any. AI involves the panel once more in order to assess the criteria. AI subsequently submits the final accreditation report to the Accreditation Council. The report with the consultation response, which has been considered in the report, forms the basis of the Council s decision. All documents from the institution are submitted to the Council, in the event that questions arise in connection with Council members reading of the accreditation report. Accreditation reports are published on the AI website prior to the Council meeting. Once the Accreditation Council has made a decision on accreditation, The Council informs the institutions of the accreditation decision in a letter. Immediately thereafter, the Council publishes the formal decision including the accreditation report on the website at 40

41 ESG 2.5 Criteria for outcomes STANDARD: Any outcomes or judgements made as the result of external quality assurance should be based on explicit and published criteria that are applied consistently, irrespective of whether the process leads to a formal decision. GUIDELINES: External quality assurance and in particular its outcomes have a significant impact on institutions and programmes that are evaluated and judged. In the interests of equity and reliability, outcomes of external quality assurance are based on pre-defined and published criteria, which are interpreted consistently and are evidence-based. Depending on the external quality assurance system, outcomes may take different forms, for example, recommendations, judgements or formal decisions. The accreditation criteria are laid down by the Minister for Higher Education in the Accreditation Order. The criteria are shown in the box below. Programme Accreditation - Criteria Criterion I: Demand and relevance Criterion II: Knowledge base Criterion III: Goals for leaning outcomes Criterion IV: Organisation and completion Criterion V: Internal quality assurance and development From the Accreditation Order, Order no. 745 of 24 June 2013 and Order no. 852 of 3 July 2015 Institutional Accreditation - Criteria Criterion I: Quality assurance policy and strategy Criterion II: Quality management and organisation Criterion III: The programmes' knowledge base Criterion IV: Programme levels and content Criterion V: Programme relevance From the Accreditation Order, Order no. 745 of 24 June 2013 and Order no. 852 of 3 July 2015 The five criteria were adopted in December 2013 on the basis of a review process, which is described in section ESG 2.2. The five criteria have been applied to the accreditation rounds from July 2014 and going forward. See also annexes D, E and F. 41

42 Programme Accreditation In Criterion I, the institution must account for the demand for the study programme. This concerns, among other things, continuous involvement of relevant stakeholders in the development of the study programme, taking into account the job opportunities for future graduates. Note that this criterion is not assessed by AI in an accreditation of a new study programme, since this is decided upon in RUVU. In Criterion II, the institution must describe its research environment/knowledge base in relation to the study programme. It must be demonstrated, among other things, that there is coherence between the knowledge base and the content of the study programme. In addition, the institution must document the quality and strength of the knowledge base. In Criterion III, the institution must document that the level descriptions of the study programme (learning outcomes) live up to the level descriptions of the Qualifications Framework and that the structure and admission of students are coherent with the learning outcome goals. In Criterion IV, the institution must document a number of different elements, e.g. the coherence and structure in the study programme and the pedagogical and student-centred learning approach. In Criterion V, the institution must document that they have continuous and systematic quality assurance. Institutional Accreditation In Criterion I, the institution must demonstrate that it has a formally adopted quality assurance policy and strategy for strengthening and developing quality and for ensuring the relevance of the programmes and the local provision of programmes on an ongoing basis. In Criterion II, the institution must demonstrate that its quality assurance is anchored at management level and is organised and performed in such a way as to promote development and maintenance of an inclusive quality culture that supports and furthers the quality and relevance of programmes. In Criterion III, the institution must demonstrate that it has a practice which ensures that programmes and teaching are always founded on a knowledge base that corresponds to that of programmes of the given type at the given level and provides a firm basis for achieving programme goals. In Criterion IV, the institution must demonstrate that it has a practice which ensures that programmes have an appropriate level, an academic content and an educational quality that supports students' learning and the achievement of programme goals. 42

43 In Criterion V, the institution must demonstrate that it has a practice which ensures that new and existing programmes reflect the needs of society and are continually adapted to social developments and the changing needs of the Danish labour market. 43

44 ESG 2.6 Reporting STANDARD: Full reports by the experts should be published, clear and accessible to the academic community, external partners and other interested individuals. If the agency takes any formal decision based on the reports, the decision should be published together with the report. GUIDELINES: The report by the experts is the basis for the institution s follow-up action of the external evaluation and it provides information to society regarding the activities of an institution. In order for the report to be used as the basis for action to be taken, it needs to be clear and concise in its structure and language and to cover context description (to help locate the higher education institution in its specific context); description of the individual procedure, including experts involved; evidence, analysis and findings; conclusions; features of good practice, demonstrated by the institution; recommendations for follow-up action. The preparation of a summary report may be useful. The factual accuracy of a report is improved if the institution is given the opportunity to point out errors of fact before the report is finalised. The accreditation reports have a clear and well-organised structure. The recommendation and overview form of the criteria assessments are placed at the beginning of the report, which provides a quick overview of the assessment of the study programme. In addition, the accreditation reports are published prior to the decision made by the Accreditation Council. It is important to note that, although reports and decisions are published and hence accessible for the wider society the accreditation reports are primarily intended as a solid decisionmaking platform for the Accreditation Council and as feedback to the institutions on the quality and relevance of their study programmes. The reports may therefore appear somewhat less readily accessible to a wider circle of stakeholders such as students, citizens and journalists. All programme accreditation reports are structured in a template. The template has been adjusted on several occasions as a consequence of adjusted criteria, but also to ensure as clear and concise presentation as possible of the assessments made. Assessments contain: a recommendation; basic information; competency profile of the study programme; structure of the study programme; information on the accreditation panel; assessment of each of the criteria; timetable; the case processing etc.; assessment. 44

45 All institution accreditation reports are also structured in a template. Reports contain: introduction; recommendation and summary of assessments; information on the panel; basic portrait of the institution; assessment of each of the criteria; methodology; audit trails; case processing and timetable; schedules for site-visits; key indicators. AI writes all the accreditation reports, and the accreditation panel receives the reports for review and approval. Since the panels are not directly involved in the writing process, it is very important for AI that the panels can recognise their assessments in the final reports. For institutional accreditation, the ongoing dialogue with the panel is supplemented with a final meeting when a draft of the report is ready. This allows the panel to discuss the phrasing of the overall assessment, of each criterion and possible recommendations to the institution. The 2015 panel survey conducted by AI confirms that this is the case. For institutional accreditation, 94 percent (31 persons) answered that the report to a very high extent or to a high extent is in line with the panel s assessments. For accreditation of existing programmes, the corresponding percentage is 87 percent (72 persons), and for accreditation of new programmes, the corresponding percentage is 89 percent (60 persons). When the Accreditation Council has made its decision, the Council publishes the accreditation report, along with the letter of approval, on its website. The letter of approval contains the decision and the reasons behind them, as well as information on the length of the accreditation period. The reports and the letter containing the decision are entered into a database on the website, where users can carry out searches according to various search criteria such as institution, main subject area, year, application type and decision. The reports for programme accreditation contain no recommendations for further development. The reports for institutional accreditation only contain recommendations for further development in the case of a positive institutional accreditation. The main reason for this is that the reports for conditionally positive accreditations and refusals of accreditation are very explicit in substantiating the arguments for the decisions. It is therefore already clear to both the Accreditation Council and the institutions which specific criteria and areas are to be improved to receive a positive accreditation. Furthermore, the institutions are autonomous, and they should be able to choose their own, relevant solutions to the problems identified in the reports. Finally, recommendations in reports for conditionally positive and refusals of accreditations could in some cases present a legal problem, since it may not be clear to the institution whether or not it could comply with the accreditation criteria by only following the recommendations. 45

46 ESG 2.7 Complaints and appeals STANDARD: Complaints and appeals processes should be clearly defined as part of the design of external quality assurance processes and communicated to the institutions. GUIDELINES: In order to safeguard the rights of the institutions and ensure fair decision-making, external quality assurance is operated in an open and accountable way. Nevertheless, there may be misapprehensions or instances of dissatisfaction about the process or formal outcomes. Institutions need to have access to processes that allow them to raise issues of concern with the agency; the agencies, need to handle such issues in a professional way by means of a clearly defined process that is consistently applied. A complaints procedure allows an institution to state its dissatisfaction about the conduct of the process or those carrying it out. In an appeals procedure, the institution questions the formal outcomes of the process, where it can demonstrate that the outcome is not based on sound evidence, that criteria have not been correctly applied or that the processes have not been consistently implemented. It should be borne in mind that the institutions are always given the opportunity of a formal hearing in the following steps of the accreditation processes: Before the establishment of the rotation plan: The institutions decide when they are ready for an institutional accreditation The institutions can comment on the rotation plan for the accreditation of existing study programmes. Before setting down the accreditation panel: The institutions are given the opportunity to point to relevant academic areas and labour markets. The institutions can comment on the composition of all panels. Before submitting the report to the Accreditation Council: The institution can comment on the draft report. After the Council decision: If the Council decides against the accreditation report s recommendation, the institution is given the option of another formal hearing. This only applies in situations where the decision is to the disadvantage of the institution or if other specific circumstances have to be considered. 46

47 The decisions by the Accreditation Council cannot be referred to other administrative authorities, which means that the Minister for Higher Education or other parties cannot affect or reverse the Council s decision concerning accreditation. The independence of the Council in terms of accreditation is thus ensured, as the system acknowledges that the Council is the only body with the requisite professional capability to make decisions on accreditation. Generally, it is not possible within the public administration in Denmark to lodge appeals against decisions made by professionally independent bodies. However, the Act opens allows for appeals against legal errors and omissions in connection with the process to be referred to the Ministry of Higher Education. The possibility of making appeals is to ensure that all decisions are made on a sufficiently detailed decision-making platform, in a fair and equitable manner, and that institutions are treated fairly and equally in the accreditation process. 47

48 ESG Part 3 Compliance by The with the ESG for Quality Assurance Agencies ESG 3.1 Activities, policy and processes for quality assurance STANDARD: Agencies should undertake external quality assurance activities as defined in Part 2 of the ESG on a regular basis. They should have clear and explicit goals and objectives that are part of their publicly available mission statement. These should translate into the daily work of the agency. Agencies should ensure the involvement of stakeholders in their governance and work. GUIDELINES: To ensure the meaningfulness of external quality assurance, it is important that institutions and the public trust agencies. Therefore, the goals and objectives of the quality assurance activities are described and published along with the nature of interaction between the agencies and relevant stakeholders in higher education, especially the higher education institutions, and the scope of the agencies work. The expertise in the agency may be increased by including international members in agency committees. A variety of external quality assurance activities are carried out by agencies to achieve different objectives. Among them are evaluation, review, audit, assessment, accreditation or other similar activities at programme or institutional level that may be carried out differently. When the agencies also carry out other activities, a clear distinction between external quality assurance and their other fields of work is needed. The quality assurance methods applied by the Accreditation Council and AI have been described elsewhere in this report. The list below is a summary: - section ESG how the interaction with stakeholders is carried out. - section ESG 2.3 o that processes and methods in connection with accreditations are pre-defined and publicly available; o how the Council and AI ensure consistency in their assessments and decisions; o that follow-up procedures are applied by the Council and AI. - section ESG how panels are appointed and site visits are organised. - section ESG that the judgements made are based on published and explicit criteria. - section ESG how the reporting is organised, the involvement of the panel in the reporting and how reports are published. As described in the section concerning Part 2 of the European standards and guidelines, the Accreditation Council and AI have had strong focus on the requirements set out in the European 48

49 standards and guidelines. This is reflected in the basis of and the work with quality assurance of the institutions and the study programmes. The tables below show the decisions concerning the institutions and existing and new study programmes accredited between 2007 and These tables show a consistent level of activities, and they show that the number of programme accreditations has decreased with the introduction of institutional accreditation in July Table 1: Decisions concerning existing study programmes calculated as of 30 September Year No. of positive accreditations No. of conditional positive accreditations No. of refusals of accreditation Total number, Table 2: Decisions concerning new study programmes calculated as of 30 September Year No. of positive accreditations No. of refusals of accreditation Total number,

50 Table 3: Decisions concerning institutional accreditations calculated as of 30 September Year No. of positive accreditations No. of conditional positive accreditations No. of refusals of accreditation Total number, January With the new Accreditation Act in 2013, not all programmes are to be accredited. However, as mentioned in section ESG 2.1, programme accreditation is still carried out for institutions that have not yet been accredited, but on a lesser scale. This explains the decline in the number of existing programme accreditations from 2014 and forward. To ensure the impact of the lesser number of programme accreditations, on the basis of a risk-based analysis, AI decided to focus on the larger programmes with a high number of students and/or on faculties and institutions that have received a relatively high number of conditionally positive accreditations or refusals of accreditations in the past. 50

51 ESG 3.2 Official Status STANDARD: Agencies should have an established legal basis and should be formally recognised as quality assurance agencies by competent public authorities. GUIDELINES: In particular when external quality assurance is carried out for regulatory purposes, institutions need to have the security that the outcomes of this process are accepted within their higher education system, by the state, the stakeholders and the public. In 2007, in pursuance of the Accreditation Act, AI was established as a professionally independent body within the state administration. This was confirmed in the renewed Accreditation Act from The Institution is subject to the state rules and regulations, including the Danish Public Administration Act, the Danish Access to Public Administration Files Act ( Offentlighedsloven ) and the Ministry of Finance s Budget Guidelines ( Finansministeriets Budgetvejledning ). The Accreditation Council and AI are not subject to the power of instruction of the Minister in relation to accreditation issues. It should furthermore be noted that a broad political agreement in the Danish Parliament backed the Accreditation Act from 2007, and the Accreditation Act from 2013 was unanimously approved. The Accreditation Act should be seen in relation to a number of reforms of the higher education legislation. This concerns, in particular, the Institution Act from 2003, the new decentralised governance concept for the institutions under the auspices of the Ministry of Higher Education, as well as the tertiary artistic education acts of 2000 and All these law reforms result in increased responsibility for the educational institutions for their own systematic quality assurance and improvement. 51

52 ESG 3.3 Independence STANDARD: Agencies should be independent and act autonomously. They should have full responsibility for their operations and the outcomes of those operations without third party influence. GUIDELINES: Autonomous institutions need independent agencies as counterparts. In considering the independence of an agency the following are important: Organisational independence, demonstrated by official documentation (e.g. instruments of government, legislative acts or statutes of the organisation) that stipulates the independence of the agency s work from third parties, such as higher education institutions, governments and other stakeholder organisations; Operational independence: the definition and operation of the agency s procedures and methods as well as the nomination and appointment of external experts are undertaken independently from third parties such as higher education institutions, governments and other stakeholders; Independence of formal outcomes: while experts from relevant stakeholder backgrounds, particularly students, take part in quality assurance processes, the final outcomes of the quality assurance processes remain the responsibility of the agency. Anyone contributing to external quality assurance activities of an agency (e.g. as expert) is informed that while they may be nominated by a third party, they are acting in a personal capacity and not representing their constituent organisations when working for the agency. Independence is important to ensure that any procedures and decisions are solely based on expertise. According to the Accreditation Act, AI is an independent institution within the public administration. Independent means that AI is operationally independent of the Ministry of Higher Education as well as of the institutions and other ministries and stakeholders. Procedures and methods The Accreditation Act stipulates that the Minister for Higher Education appoints the Executive Director of AI following a recommendation from the Accreditation Council. In respect of its accreditation activities, the management of AI do not report to the Minister and is thus not subject to the power of instruction of the Minister. This means, among other things, that AI is responsible for laying down its own procedures and methods for accreditation. The revisions of the guidelines mentioned in section ESG 2.2 have been prepared in collaboration with the institutions, but final editing is the sole decision of AI. In connection with AI s appointment of experts for the accreditation panels, a procedure has been prepared to ensure the independence of the experts. The selection of experts is based on the principles specified in the Memo on the recruitment for accreditation panels. AI carries the 52

53 sole responsibility for recruiting experts to the panels, and prior to decisions, all proposals for panel members are submitted to the institutions for hearing in order to identify conflicts of interest, if any. On the training days for panel members, AI focuses on the roles, responsibilities and tasks of the panels. AI points out to the panel members that they must carry out their assessment within the framework of the guidelines. AI is responsible for ensuring consistency across study programmes, which may mean that the accreditation panels will have to adjust their assessments or intensify their argumentation for an assessment (see also section ESG 2.4). Independence of the Accreditation Council The Accreditation Council has various mechanisms in place to ensure independence in the Council's decisions. First of all, in accordance with the rules of procedure of the Council, a member of the Council is obliged to inform the Council if, in connection with a specific accreditation case, there are matters which may give rise to doubt as to the independence of the member. In the event of a conflict of interest, the member in question leaves the room while the case is being considered. The conflict of interest is recorded in the minutes. Secondly, the decisions made by the Accreditation Council cannot be referred to other administrative authorities, and the Minister for Higher Education or other parties cannot affect or reverse the Council s decisions concerning accreditation. However, as mentioned, the Act allows for appeals against legal errors and omissions in connection with the process to be referred to the Ministry of Higher Education. 53

54 ESG 3.4 Thematic analysis STANDARD: Agencies should regularly publish reports that describe and analyse the general findings of their external quality assurance activities. GUIDELINES: In the course of their work, agencies gain information on programmes and institutions that can be useful beyond the scope of a single process, providing material for structured analyses across the higher education system. These findings can contribute to the reflection on and the improvement of quality assurance policies and processes in institutional, national and international contexts. A thorough and careful analysis of this information will show developments, trends and areas of good practice or persistent difficulty. According to the Accreditation Act, AI must comply with international standards, and must also collect national and international experience of relevance to accreditation. (Accreditation Act, 2) AI is therefore under an obligation to conduct summary analyses of work on accreditation and disseminate the most important results of this work. In recent years, several thematic analyses have been carried out. In 2015 the following projects were conducted: Small campuses: the challenges and strengths regarding the quality assurance of programmes at small campuses situated in another city that the main campus External lecturers: the use of external lecturers and the quality assurance of their pedagogical competencies and integration of these into programmes Massive Open Online Courses (MOOCs): how MOOCs are used at the HEIs and how quality assurance of online courses can be carried out The institutions knowledge of future labour market needs: how the institutions receive and use input from the labour market and other sources to develop their programmes These projects have been adopted following an external as well as an internal idea process. The plans for the analyses have been presented to the HEI rector s conference, the students in STAR, and to employer and trade representatives in consultation meetings. The projects cover areas and themes that are relevant and challenging for the institutions, and some of them have arisen in the accreditation processes. The aim of the projects is to disseminate knowledge and ideas for further improvement. The public can monitor the results of the analyses and the general accreditation work on an ongoing basis. The analyses will be published on the AI website when completed. 54

55 ESG 3.5 Resources STANDARD: Agencies should have adequate and appropriate resources, both human and financial, to carry out their work. GUIDELINES: It is in the public interest that agencies are adequately and appropriately funded, given higher education s important impact on the development of societies and individuals. The resources of the agencies enable them to organise and run their external quality assurance activities in an effective and efficient manner. Furthermore, the resources enable the agencies to improve, to reflect on their practice and to inform the public about their activities. Since its start-up in 2007, AI has expanded its staff on a regular basis. In 2013, a merger between ACE Denmark and parts of EVA meant that AI reached its current staff of 41 employees (38.5 full-time equivalents (FTEs)). AI regards this as sufficient capacity to handle the accreditations of institutions and programmes as well as conducting the thematic analyses. Funding for AI is a combination of government funding and to a much lesser degree income is also generated from accreditation of study programmes abroad which are not covered by the Accreditation Act. The numbers below are thus divided into government funding and other generated income: Year Funding DKK (millions) Other income DKK (millions) Total income DKK (millions) FTEs ,6 0,1 36,7 38, ,2 0,0 30, ,3 0,4 29, ,9 0,1 25, ,6 0,1 26,7 34 AI s 38.5 FTEs (41 employees) are distributed between the Administrative Unit (communication, administration, HR and law), the two Professional Secretariats (accreditation of institutions and study programmes, development of procedures and methods) and the Council Secretariat (secretariat duties for the Accreditation Council). See also Part 1 for a more detailed description of the responsibilities of these units. The employees are distributed between the secretariats and the Administrative Unit as follows: The Administrative Unit (7): the Executive Director, the Head of Communications, the Chief Financial and IT officer, the Legal Officer, the Executive Secretary, one Senior Administrative Officer, one Financial and Travel Coordinator. 55

56 The Area for Professional, Vocational and Maritime Institutions (14): the Director of Operations, two Senior Advisors and eleven Accreditation Officers. The Area for Universities and Educational Institutions of Arts and Culture (14): the Director of Operations, two Senior Advisors, two Special Advisors and nine Accreditation Officers. The Council Secretariat (6): the Head of Division, two Special Advisors and three Heads of Section. The educational backgrounds of the employees are as follows: four employees educated within administration and service; 37 employees with a long-cycle higher education. In addition, there are 11 student assistants, who handle support functions across the secretariats and the Administrative Unit. The student assistants have a working week of fifteen hours. As part of continuous professional upgrading of the staff, qualifying courses have been held in relation to the performance of accreditation duties. These include courses for relevant employees on teamwork and process management, and courses concerning administrative law for relevant employees. The management systematically reviews the individual employee s needs for competency development at the annual employee performance interviews (MUS). Some accreditation consultants have completed the project manager training programme, for example, and other employees have participated in an English training course at the Educational Unit at the Ministry of Foreign Affairs. 56

57 ESG 3.6 Internal quality assurance and professional conduct STANDARD: Agencies should have in place processes for internal quality assurance related to defining, assuring and enhancing the quality and integrity of their activities. GUIDELINES: Agencies need to be accountable to their stakeholders. Therefore, high professional standards and integrity in the agency s work are indispensable. The review and improvement of their activities are on-going so as to ensure that their services to institutions and society are optimal. Agencies apply an internal quality assurance policy which is available on its website. This policy ensures that all persons involved in its activities are competent and act professionally and ethically; includes internal and external feedback mechanisms that lead to a continuous improvement within the agency; guards against intolerance of any kind or discrimination; outlines the appropriate communication with the relevant authorities of those jurisdictions where they operate; ensures that any activities carried out and material produced by subcontractors are in line with the ESG, if some or all of the elements in its quality assurance activities are subcontracted to other parties; allows the agency to establish the status and recognition of the institutions with which it conducts external quality assurance. AI has defined a mission, vision and values for the organisation. A quality policy provides the framework for assuring the quality of our processes. In addition, the performance contract, director s contract, annual review and annual report set out goals and results for the organisation. The documents have been published on the AI website. In 2013, the management started a process of redefining the values and the strategy of AI. The background was the 2013 Accreditation Act and Ministerial Order on Accreditation, as well as the ensuing merger between the (ACE Denmark) and the unit responsible for the accreditation of university colleges and vocational programmes at EVA (The Danish Evaluation Institution). The strategy process has involved all of the organization and has resulted in a redefined set of values, mission and vision (2014), a new strategy for the organization (2015) and a new quality policy (2015). AI s values should be seen in light of a development process where the values are cultural traits that the organisation should work to achieve and keep in mind. Inspired by the new ESG, the quality policy in its current form is a result of a seminar held in 2015, where employees from the four different units identified key activities and the corresponding measures taken to assure the quality of the given activities. The quality policy provides a new framework for quality assurance measures that have been in place, developed 57

58 and adjusted since Thus, it is important to stress that, although the quality policy is new, the quality assurance activities described in the policy are not. Values Constructive Inclusive Sharp Exploratory Mission and Vision Our mission is to contribute to improving the quality of higher education and to qualify democratic debate in society and the political decisions which shape the Danish higher education system. We do this through accreditations which make a difference. They identify specific problems, promote quality improvements and motivate continued development. In order to contribute optimally to quality improvements, we work on the basis of a targeted fit-for-purpose approach, and we support institutions own quality assurance work and development activities. Accreditation is thereby a suitable method to improve pivotal parts of institutions work on quality. In order to support our core task, our vision is to continually develop the accreditation task. Quality Policy AI has decided a quality policy, and this is available on AI s website. The quality policy describes how AI assures the quality of its processes. Overall responsibility for the quality policy rests with the management The quality policy contains designated areas in which AI works to assure the quality and the continuous improvement of AI s processes. The quality policy is an extension of AI s values, mission and vision, and is itself supplemented by several policies, guidelines, procedures and templates. The quality policy emphasises the following areas: An efficient administration. AI ensures that administrative processes are efficient and in line with the rules for organisations in the Danish state sector on all areas, including budgets, data management, case processing and record-keeping. A professional staff. AI ensures competent and professional staff through clear recruitment procedures, thorough introductions, cooperation, feedback and continuous development of skills. Competent experts. AI ensures that panel members are competent and free of conflicts of interest. This is ensured through recruitment principles and procedures regarding the composition of panels and thorough training. 58

59 Participation in AI s training sessions is mandatory for all panel members. The training ensures that all panel members have knowledge of the accreditation process, the higher education sector in Denmark, the role of the panel and the role of the consultants/ai. See ESG 3.4 for further details. Equal treatment of HEIs. AI ensures equal treatment of all HEIs through consistent application of predefined and published criteria, clear standards and guidelines AI ensures clear knowledge of the specific conditions that apply to an HEI/study programme through dialogue and by recruiting experts with relevant knowledge about the specific conditions that apply to an HEI or programme or subject area. Dialogue with relevant stakeholders. AI views dialogue as an important part of assuring the quality and relevance of its processes. Thus, AI maintains a strong dialogue with HEIs, students, the Ministry of Education, labour market representatives, national and international networks for evaluation and quality assurance in higher education such as EVA, NOQA, ENQA, ECA, INQAHEE (see also ESG 2.2.) Internal and external feedback mechanisms. AI views internal and external feedback as an important part of the accreditation process and has several mechanisms in place to ensure this. The procedures for internal and external feedback are described in detail below. Internal feedback The accreditation processes are continuously discussed and evaluated at the bi-weekly PEM and UNIK meetings, where the accreditation consultants update each other on the process and specific challenges they are dealing with. New elements in the accreditation process are evaluated. Minutes from the meetings are made available on AI s intranet. New and existing study programmes AI has well-established procedures for ensuring the quality of accreditation reports through systematic internal feedback mechanisms. Two experienced accreditation consultants always provide feedback to the individual consultant on each accreditation report. The feedback process is anchored in a feedback group. The group meets on a regular basis to discuss issues of principle. The director of operations is a member and takes part in these meetings and is responsible for ensuring consistency in the application of the assessment principles. Minutes from the meetings in the feedback groups are shared with all accreditation consultants and discussed at unit meetings. Cross-unit meetings with all staff from PEM and UNIK are held on a regular basis and matters of principle are discussed at these meetings. Moreover, the Executive Director and the directors of operations from PEM and UNIK hold meetings at which accreditation reports recommending conditional positive accreditation or refusal of accreditation are discussed with relevant staff. 59

60 Institutional Accreditation The director of operations from either PEM or UNIK participates as an observer in the panel s preparatory meetings and at all visits at the institutions, and is thus able to provide detailed feedback to the team and ensure consistency in the institutional accreditations. Every report is the subject of several meetings between the team responsible for the institutional accreditation, the director and the heads of PEM and UNIK, where the report and the documentation are discussed thoroughly. During and following the first round of institutional accreditations in 2015, the process was the target of several evaluations and discussions at PEM and UNIK meetings. External Feedback AI has a well-established practice of using external feedback, including systematic evaluations of processes. The systematic evaluations are mainly directed towards members of the accreditation panels and persons involved in accreditation processes at the HEIs. Prior to 2013, systematic evaluations of the accreditation processes were carried out in both the university sector and the professional and vocational sector. In the university sector, evaluations aimed at the universities themselves and persons specifically involved with the accreditations were carried out in 2010, 2011 and In addition, two evaluations aimed at the members of the accreditation panels were carried out in 2011 and For the professional and vocational education sector, annual evaluations on accreditation of new study programmes and biannual evaluation on accreditation of existing study programmes were carried out. These were also aimed at both HEIs and panel members. These evaluations contained questions on the different steps of the accreditation process, the dialogue as well as questions concerning overall satisfaction with the process (experts) and whether the accreditation process led to any changes in the accredited programmes. In 2013, following the Accreditation Act, AI conducted a small-scale evaluation based on interviews with 16 key representatives from the eight universities. Following the transition to institutional accreditations and the first round of accreditations in the new system, AI gathered systematic external feedback on all five accreditation processes in 2015, involving HEIs and panel members. The evaluation of institutional accreditations was generally very positive on almost all parts of the process. However, both institutions and panel members noted that the amount of documentation requested for the audit trails was large. For the HEIs the workload compiling the documentation became quite heavy. For many panel members, the documentation seemed excessive and, in some parts, less relevant. As a consequence AI will seek to strengthen the dialogue with the HEIs when requesting documentation for audit trails to ensure the relevance of the documents. Feedback from the Accreditation Council Finally, the Accreditation Council also has a role in ensuring the quality of AI. Up until the merger between ACE Denmark and the accreditation unit of EVA in 2013, the Accreditation Council reviewed the processes and methods of both operators. 60

61 As described in ESG 2.2., the Accreditation Council has been involved in the development and ongoing quality assurance of the accreditation processes on a regular basis. Examples from include a memorandum on the concept of the institutional accreditation decisions, a memorandum on the knowledge base of university colleges, a discussion of the use of quantitative data in accreditations and an inquiry on the use of panel members in accreditations. Memoranda that have been discussed by the Accreditation Council are made public and thereby contribute to the ongoing development as well as the transparency of accreditation system. ESG 3.6 requirements not addressed in the quality policy The quality policy does not address the requirements stated in ESG 3.6 that the policy outlines the appropriate communication with the relevant authorities of those jurisdictions where they operate, and allows the agency to establish the status and recognition of the institutions with which it conducts external quality assurance. This is due to the fact that AI has a well-defined national stage of operation with no cross-border accreditation activity. The Danish legislation very clearly defines 1) the status of AI in connection to the relevant authorities and 2) the status of the institutions that are subject to accreditation by AI. In addition, the requirement connected to subcontractors is not relevant, since subcontractors are not used by AI. 61

62 ESG 3.7 Cyclical external review of agencies STANDARD: Agencies should undergo an external review at least once every five years in order to demonstrate their compliance with the ESG. GUIDELINES: A periodic external review will help the agency to reflect on its policies and activities. It provides a means for assuring the agency and its stakeholders that it continues to adhere to the principles enshrined in the ESG. AI was reviewed by ENQA in 2010 and also in 2012 as a follow-up on the recommendations from the ENQA review panel in This self-assessment report forms the basis for the review in

63 63

64 The Bredgade 38 DK-1260 København K Denmark phone: [email protected] web: en.akkr.dk