KOSOVO ACCREDITATION AGENCY (KAA)

Transcription

1 KOSOVO ACCREDITATION AGENCY (KAA) SELF-EVALUATION REPORT Approved by KAA s Board on 24 October 2013 This Report is also available on-line on the Agency s website, 1

2 CONTENTS 1. Introduction Context and purpose of the Self-Evaluation Report Background to the self-evaluation process Higher Education and Quality Assurance in Kosovo Background to current situation of higher education in Kosovo Structure of Kosovo higher education Degree structures Development of Quality Assurance in Kosovo higher education Current status and role of KAA Funding of KAA KAA organization and bodies KAA s international activities KAA s accreditation activities Accreditation procedures and decisions KAA-defined standards and procedures Different types of accreditation procedures External quality assurance for study programmes Initial accreditation and licensing Reaccreditation procedures Other KAA responsibilities: Stragetic development plan of KAA The national context SWOT analysis for the Agency Strategic priorities for the quinquennium KAA Compliance with European Standards and Guidelines ESG part ESG 2.1. Use of internal quality assurance procedures ESG 2.2. Development of external quality assurance processes ESG 2.3. Criteria for decisions ESG 2.4. Processes fit for purpose ESG 2.5. Reporting ESG 2.6. Follow up-procedures ESG 2.7. Periodic reviews ESG 2.8. System-wide analyses ESG part ESG 3.1. Use of external quality assurance procedures ESG 3.2. Official status

3 ESG 3.3. Activities ESG 3.4. Resources ESG 3.5. Mission statement ESG 3.6. Independence ESG 3.7. External QA criteria and processes used ESG 3.8. Accountability procedures Conclusion List of Annexes

4 LIST OF ACRONYMS AI Administrative Instruction (from MEST) BFUG Bologna Follow-Up Group CEENQA Central and Eastern European Network for Quality Assurance ECTS European Credit Accumulation and Transfer System EHEA European Higher Education Area ENQA European Association of Quality Assurance in Higher Education EQAR European Registry of (credible) QA agencies EQF European Qualifications Framework ESG European Standards and Guidelines HE Higher Education HEI Higher Education Institution INQAAHE International Network of Quality Assurance Agencies in HE KAA Kosovo Accreditation Agency KAIP Kosovo-Austria Institutional Partnership Project on HE and Research LHE Law on Higher Education (Kosovo) MEST Ministry of Education, Science and Technology (Kosovo) NQC National Quality Council (KAA s Board) NQF National Qualifications Framework RaG Report at Glance UNMIK United Nations Interim Administration Mission in Kosovo 4

5 1. INTRODUCTION 1.1. Context and purpose of the Self-Evaluation Report Kosovo higher education is facing many challenges in its role as a promoter of economic, social and cultural development of the society, with the aim to transform it in a society of knowledge and innovation, able to compete in a global market of knowledge. But the current state of higher education in Kosovo cannot be understood without previous knowledge of its recent political developments. Among the many challenges higher education in Kosovo has faced, the largest and most important has been its struggle for existence during the segregation process in the period This period is known as the period of survival when, as the result of the segregation policy by the then Serbian regime, higher education in the Albanian language functioned as a "parallel system organized in private homes at all level of education. This served the Albanian-speaking students, teachers, professors and administrative staff who had been banned from the only university in that time (the University of Pristina), as well as from the whole school system. With many efforts and strong commitments of both academic staff and students, Albanian-speaking students continued their studies in private homes, with poorly equipped classrooms, without adequate equipment, literature, laboratories and other services. This parallel, underground education system in Albanian language functioned until the escalation of the armed conflict in March 1999.This ten-year period of isolation caused delays in the development and reforms of higher education in Kosovo in comparison with neighbouring countries and Europe as a whole. After the conflict ended in 1999, the United Nations Interim Administration Mission in Kosovo (UNMIK) was initially in charge of all areas of education, and was tasked with creating formal institutions and authorities in Kosovo, such as the Ministry of Education, Science and Technology (MEST) that was set up in With strong support from the international community present in Kosovo, higher education went through tremendous developments and reforms in an effort to catch up with the developments elsewhere in the European area. Besides the reconstruction of the education system and the reorganization of academic life, another challenge for higher education at that time resulted from the adoption of

6 the Bologna Declaration by the University of Pristina in 2001, making it one of the first universities in Europe to start with the reforms called for in the EHEA. In the spirit of the Bologna Declaration a new Law on Higher Education (No. 2003/14) was drafted in 2002 and adopted in It focuses on the necessary structural reforms and the establishment of internal and external quality assurance systems in higher education that would enable Kosovo to effectively integrate in the European Higher Education Area (EHEA). In 2008, on the basis of the Law on Higher Education (No. 2003/14) (Annex 1), the Ministry of Education, Science and Technology (MEST) made the decision to create the Kosovo Accreditation Agency (KAA) with real authority to act as a public, independent agency responsible for assessing and promoting quality in higher education in Kosovo. Since its establishment KAA has been actively involved in supporting higher education to transform itself through the implementation of reforms and the increasing awareness of the importance of quality and quality assurance. Therefore the Kosovo Accreditation Agency is seen in society as a key actor effectively promoting the development and improvement of quality in higher education. In order to effectively play its role, KAA has developed its procedures and expertise based on the best international practices in the area of higher education quality assurance. From the very beginning KAA has been committed to developing its standards and procedures in full compliance with the European Standards and Guidelines for quality assurance in Higher education (ESG), by adapting them in the national context. During its short life, KAA has placed great importance on communication with stakeholders and public opinion in order to make its work transparent and to build up public trust. At the moment, KAA is affiliate to ENQA and is a member of CEENQA and INQAAHE. A key strategic objective of the Agency is to become a full member of ENQA and EQAR. Due to the small size of the country, in full awareness of the need to avoid conflicts of interest, a golden rule of KAA has been to engage only international experts in its evaluation procedures, which has given a marked international dimension to its work and its profile. 6

7 1.2. Background to the Self-Evaluation process Since its establishment KAA has been committed to act in full compliance with the European Standards and Guidelines for quality assurance in higher education (ESG). The main objective of this Self-evaluation process is to assess KAA s compliance with the ESGs with the aim to achieve full membership in ENQA and EQAR. However, at the same time KAA wants to use the opportunity to critically analyze whether it operates effectively with regard to its main stakeholders in Kosovo and partners in Europe, and to learn what it can do to further improve its work. KAA decided therefore to undergo a type A evaluation to be coordinated by ENQA. A specific working group to draft the Self-Evaluation Report was set up in February 2013 by the Agency s Board the National Quality Council (NQC). It included representatives from the Secretariat and the Board, including both Kosovo and international members. This working group was tasked with the preparation of a draft Self-Evaluation Report (SER). This group organized regular meetings to discuss and share information during the drafting period of the SER. The involvement of stakeholders in the Self-Evaluation process is very important for KAA. The Ministry of Education and other stakeholders were informed about KAA s application for ENQA and EQAR membership and were invited to contribute to the self-evaluation. A main initiative in this direction was KAA s decision to conduct a survey for which more than 500 questionnaires (Annex 24) were distributed electronically to all categories of stakeholders. The results of this survey are presented in Chapter 5. The revised draft of the Self-Evaluation Report was sent to the National Quality Council (i.e. to the full Board of KAA) where it was further discussed. Comments were integrated in the text and the SER was formally adopted by the full Board at its meeting of 24 October

8 2. HIGHER EDUCATION AND QUALITY ASSURANCE IN KOSOVO 2.1. Background to current situation of higher education in Kosovo Kosovo has the youngest population in Europe, therefore there is a real challenge and opportunity for institutions of higher education to provide quality education and prepare the young generation for the job market. As elsewhere, quality in higher education in closely linked to quality in all other education. The education system in Kosovo extends from pre university education through higher education. All of its components are under the oversight of the Ministry of Education, Science and Technology of Kosovo. Pre-University Education in the Republic of Kosovo includes children aged 6-18 attending one of the three education levels: primary school (grades 1-5), lower secondary school (grades 6-9) and upper secondary school (grades 10-12/13). The first two levels are compulsory for all children. After completion of upper secondary school all students have to undergo the Matura exam, the organization of which is regulated by the Law on Final Examinations and State Matura Examination (No. 2008/ 03/L-018). Students who successfully pass the Matura exam have the right to enter higher education. The organization of higher education is based on the Law on Higher Education in the Republic of Kosovo (No. 04/L-037), (Annex 2) adopted in August The role and responsibilities of the Kosovo Accreditation Agency are regulated by the Articles 7 and 15 of the same law. Although Kosovo has not officially joined the Bologna Process, since the introduction of Bologna Reforms by the University of Prishtina in 2001/2002 all national legislation has served to better implement and make operational the Bologna reform objectives and principles in Kosovo. As a result of the implementation of these reforms in higher education in Kosovo today all institutions of higher education implement the ECTS credit system, curriculum reform, three-cycle degree system, student and staff mobility, diploma supplement and quality assurance system. The sole exception is an Americanstyled institution founded after the recent war. 8

9 In the effort to follow the trends in EHEA, policy makers in Kosovo have been most challenged in respecting the autonomy of HEIs and increasing the responsibility of HEIs for their quality. This is hardly unusual in nations shifting from heavy ministerial control to more independent quality assurance, and it is to be expected in a nation transitioning from a highly centralized system of control to something significantly more decentralized. Most of the fundamental reforms in higher education were done through and supported by the KAIP project, the Kosovo Austria Institutional Partnership Project on Higher Education and Research. The development and reform of higher education at National level is supported by a few strategic documents, implemented by MEST. The Strategy for Development of Higher Education in Kosova (Annex 22) and the Kosovo Education Strategic Plan (Annex 25) are the two main strategic national documents that support the development of higher education in the Republic of Kosovo. One of the main goals of these two documents regarding Kosovo higher education is to formally become part of the Bologna Process, the European Higher Education Area and the European Research Area. The research activities in HE are regulated by the National Research Programme and the Law on Scientific Research Activity in Republic of Kosovo (No. 2004/42), two additional documents that support the scientific and research activity at national level, from which higher education institutions benefit regularly to support their research programmes, their publications and the mobility of academic staff and students. The National Qualifications Law (No. 03/L-060), and the National Qualifications Framework, which were developed to fully integrate the European Qualifications Framework, are two other legal and strategic documents that regulate the qualifications of higher education in the Republic of Kosovo. Last but not least, NARIC Kosovo is the National Academic Recognition Information Centre for academic recognition of diplomas and study period, undertaken in other States Structure of Kosovo higher education The higher education system in Kosovo operates through public universities and private higher education institutions (colleges, institutes, higher professional schools). Only accredited institutions of higher education can offer study programmes that lead to a degree or diploma. 9

10 The interest of the young population in Kosovo for enrolment in higher education has risen over the last 10 years. So have the numbers of opportunities available to these students. Starting in the year 2003, the number and the types of higher education institutions in Kosovo changed very quickly. Within a short time a considerable number and range of private institutions were opened, almost all with no internal system of quality assurance. In 2008, before the Kosovo Accreditation Agency become functional, MEST appointed the British Accreditation Council (BAC) to conduct an ad hoc external audit at all 30 private providers of higher education in Kosovo that were operating at that time. The report on this global audit was published in July 2008 (Annex 21). From the thirty private providers of higher education presented in Kosovo, 29 failed to pass the evaluation, particularly as related to their claim to be a university. As a result of the findings during this audit, BAC recommended that many of these providers should be transformed into vocational colleges. As such, most of these institutions reapplied in 2009 to undergo external evaluation at institutional and programme level by Kosovo Accreditation Agency. A total number of 16 HEI s were evaluated by KAA in that process. As a result, at the institutional level most of the private providers are accredited as colleges. Regarding the establishment of new HEIs in the Republic of Kosovo, public providers of higher education may only be created by decision of the Ministry that must be ratified by the Assembly. They have to undergo accreditation by KAA within twelve month after establishment. A new private provider of higher education aspiring to offer higher education qualifications must undergo a preliminary (ex ante) KAA process at the institutional and programme level before it is licensed by MEST. The Statutes of all institutions titled University must be ratified by the Assembly, while the statutes of other types of HEI have to be approved solely by the Minister of Education. Until 2009 the only university in Kosovo was the public University of Prishtina. During the last four years, in order to increase the inclusion of the young population in higher education, the Government of Kosovo decided to expand the number of public universities. Today the accredited HE sector in Kosovo has 3 public universities, 25 private institutions of HE (22 denominated as colleges, 2 higher professional schools and one Academy of arts). Beside this, there is a public university in north Mitrovica and three higher professional schools where the language of instruction is Serbian. These four institutions continue to operate under Serbia's legal framework and so far were not subject of evaluation by KAA. 10

11 Recently the Government of Kosovo announced the establishment of three other public universities in three cities in Kosovo: Gjilan, Gjakovë and Mitrovica. In Kosovo the Law on Higher Education classifies the higher education institutions in the following categories: A University offers education and scientific research/artistic programmes. It can offer professional programmes too. Universities give diplomas and degrees at all levels including the doctoral level. There are 3 accredited and one not (yet?) accredited public university in Kosovo with a total number of students, of which study at the University of Prishtina. University Colleges offer programmes with professional orientation that gives diplomas and degrees including the doctoral level. They can offer academic oriented programmes too. At the moment there is no university college accredited in Kosovo. Colleges offer professional or academic programmes at Bachelor and Master level. Currently there are 22 accredited private colleges in Kosovo with students. Two of them offer study programmes in English language. Only two out of 22 colleges are for non profit, the others are for profit. Higher professional schools offer professional courses in one or more professional fields. There are 2 private accredited schools and three other non accredited in Kosovo, with a total number of 3364 students. Academies provide higher education and creative activity in specific areas of arts, sports or other professional areas. So far only one Academy is accredited. It has 326 students. The Ministry of Education, Science and Technology of Kosovo is responsible for planning the development of higher education in Kosovo, regulating public and private providers of higher education and approving their Statutes, allocating funds to higher education providing for purposes of teaching and research in the public interest, on the basis of accreditation decision licensing private providers of higher education and providing assistance with legislation. The Minister of Education appoints the Members 11

12 of the National Quality Council (KAA s Board). However, the Ministry has not the right to change accreditation decisions or to interfere in them in any way. The Kosovo Assembly ratifies the Law on Higher Education and Statutes of public universities. The appointed members of the National Quality Council (NQC) by the Minister also have to be ratified in the Kosovo Assembly. The relevant bodies in HE system in Kosovo 12

13 Table 1: Number of students in both public and private institutions of higher education in Kosovo Number of students in universities, colleges, professional schools and academies Public Sector Private Sector Universities Faculties Professional Schools* Colleges Professional schools Academies *institutions of higher education which were not subject of evaluation by KAA Chart 1: Number of universities, colleges, faculties, professional schools and academies in the Republic of Kosovo

14 University of Prishtina University of Prizren Faculty of Islamic Studies Private Bearers of Higher Education University of Peja University of Mitrovica Public professional schools Chart 2: Number of students at both sectors in the Republic of Kosovo in Degree structures Following the Bologna reform, the majority of study programmes at public and private higher education institutions in Kosovo implement the three cycle studies (3+2+3), with exception of some disciplines, which follow scheme. Higher education is organized as following: first level three (3) to four (4) years of studies by which the student reaches 180, respectively 240 ECTS, and obtains Bachelor Diploma. second level one (1) to two (2) years of studies after completion of the first level by which the students reaches 60, respectively 120 ECTS and the student obtains Master Diploma. third level programme of doctorate studies with academic and independent research scientific character. 14

15 Integrated studies 300 ECTS Medicine,Dentistry,Vet.medicine Architecture, Pharmacy, etc. The scheme of higher education in the Republic of Kosovo 3rd level of Higher education Doctorate Studies PhD/Dr 180 ECTS Minimum 3 years 2nd level of Higher education MSc, MA Academic or professional 1-2 years, ECTS 1st level of Higher education 2 years professional studies 120ECTS Bachelor BSc, BA Academic or professional 3-4 years ECTS Secondary education - Matura exam Undergraduate level: Bachelor programmes are offered in all types of HEI and takes 3-4 years ( ECTS credits). A bachelor thesis is required at most of the Bachelor programmes. Depending on the discipline, accredited bachelor programmes can lead to a Bachelor of Sciences (BSc.), Bachelor of Arts (BA), Bachelor of Law (LL.B.), Bachelor of Engineering (B.Eng.) Bachelor of Education (B.Ed.) or Bachelor of Fine Arts (B.F.A.) Graduate level: Master programmes last 1-2 years ( ECTS credits) depending on the length of Bachelor studies. Master programmes can be provided at Universities, University Colleges and Colleges. A Master thesis work is required at all master programs. Accredited Master programs, depending on the study discipline, can lead to 15

16 Master of Science (M.Sc.), Master of Arts (M.A.), Master of Laws (L.L.M), Master of Engineering (M. Eng.) or Master of Fine Arts (M.F.A.). Postgraduate or doctoral level: are programmes with academic and independent research scientific character. They can be offered at Universities and University colleges. So far they are accredited only at one public university. The length of this cycle is at least 3 years or 180 ECTS. All students that have collected at least 300 ECTS in previous studies can register to doctoral studies. Exceptions to this study scheme are studies in medicine, dentistry, pharmacy and veterinary medicine which are organized in integrated form ( ECTS). After completion of the studies students qualify to register doctoral studies. There are also a few examples that some colleges offer a two year professional study programmes (120 ECTS), which lead to a diploma and qualifications equivalent to level 5 of National Qualification Framework as well as the European Qualifications Framework. All HEIs in Kosovo implement the ECTS system which is regulated by an Administrative Instruction issued by MEST. The annual full-time student workload is estimated at 1500 hours, making one ECTS credit point equal to 25 hours of planned instruction. The methodology, criteria and procedures for student s assessment are regulated by the Statutes of HEI and are published. Since introduction of Bologna Declaration by HEI in Kosovo, the student assessment is organized not only at the end of semester but several times during the semester. The midterm assessment usually is in a test form or seminars, whereas final exam can be in oral or written form. The grading scheme is from 5 to 10, based on the score student get out of 100%. The mode of organization of studies is regulated by the Statutes of the providers of HE qualification. It can be organized full-time, part-time, by distance learning and the combination of any of these ways of study. 16

17 Chart 3: Number and type of study programmes offered in accredited HEI s in Kosovo in Development of Quality Assurance in Kosovo higher education The beginnings of quality assurance in Higher Education in Kosovo are found in the Law on Higher Education (No. 2002/3), (Annex 1). According to Section 4.3 of this law the Ministry shall by administrative instruction provide for the establishment of the Kosovo Accreditation Agency (KAA) charged with promoting the quality of higher education in Kosovo. The Administrative Instruction on Establishment of Kosovo Accreditation Agency (No. 11/ 2004), (Annex 3), was developed by MEST and entered into force in February On 27 March 2008 the Kosovo Accreditation Agency was established. It started with first accreditation activities in October of the same year. With respect to internal quality assurance within HEIs, the first QA unit in a HEI was established in In 2009 the MEST issued an Administrative Instruction which regulates the establishment of quality assurance units at all HEI s in the Republic of Kosovo. The quality assurance unit at HEI is under supervision of the Vice rector for teaching or Vice rector for QA. It usually has a QA coordinator who is responsible to coordinate conducting on regular basis student evaluation of the courses, surveys, SWOT analyses and preparing self-assessment. QA coordinators are contact persons with KAA during the accreditation period. They participate in meetings with KAA, 17

18 training seminars and other activities relevant for QA. In order to support the establishment and development of QA units within HEIs, KAA participated in a number of Tempus projects focused on this theme. With respect to external quality assurance, the Administrative Instruction on Establishing the Kosovo Accreditation Agency (No. 11/2004), (Annex 3) specifies the establishment, role and responsibilities of the Kosovo Accreditation Agency (KAA). Pursuant to the Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) (Annex 2) and the Administrative Instruction on Establishing the Kosovo Accreditation Agency (No. 11/ 2004), (Annex 3), the KAA shall be governed by a Board called the National Quality Council (Regulation of National Quality Council) (Annex 7), a collegial body that has overall responsibility for its operations. The first members of the National Quality Council were appointed by the Minister of Education during 2006 and Since its establishment, NQC has always included three international experts. In 2007 NQC started to develop the documents, standards and procedures for accreditation as well as to provide training seminars for existing institutions of higher education in order to inform them about the standards of quality assurance in Higher education and to prepare for the self assessment. Despite all these efforts of National Quality Council to start accreditation process, the establishment of KAA was delayed until 2008 when the Minister of Education gave it power to carry out its work. Therefore, the period from 2003 to 2008 was characterized by the establishment of a big number of private institutions which by the spring 2008 included over 30 so called universities without any quality control system in place. This chaotic situation in higher education during this 5 years period caused a decrease of public trust in higher education to the lowest possible level. Because of this situation and the delay to make viable the national agency for accreditation, in spring 2008 the new Kosovo Government decided to appoint the British Accreditation Council (BAC) to conduct an ad hoc external audit at all private providers of higher education licensed by MEST during The BAC s Kosovo Accreditation Project Report 2008 (Annex 21) has shown that none, apart from one, of 31 evaluated institutions met the legal requirements to be a university, specified in the Law on Higher Education (No. 2002/3) (Annex 1). Based on this evaluation, in 2008 only one private institution of higher education was licensed to register new students. 18

19 In autumn 2008 with the appointment of the first acting director and two administrative staff, KAA became fully operational and implemented the process the agency earlier outlined. The KAA staff took the duty to continue with development of the procedures and to start with the first site visits at institutions of higher education. The Administrative Instruction on the Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 2/2013), (Annex 4) developed by MEST on recommendation of KAA, served HEIs as a guideline to prepare the application (Self-Evaluation Report) for the first accreditation process. During 2009/2010 KAA evaluated all private institutions of higher education and their study programmes as well as some of the faculties of the University of Prishtina. The data of this process are presented in Chart 4, 6 and 7. Due to the many shortcomings that expert panels found at Kosovo institutions, particularly private ones, the first accreditation was usually given for only one year. After this first accreditation round, KAA has conducted two more accreditation such rounds. In the last process a total of 29 institutions of higher education with 258 study programmes were evaluated and 206 were accredited/ reaccredited. (Please see Chart 6 and 7). Although the independence of the Agency was already provided in the Law on Higher Education (No. 2002/3) approved in 2003, (Annex 1) the first NQC s accreditation decisions were endorsed by the Government of Kosovo or by the Minister of Education in The new Law on Higher Education in the Republic of Kosovo (No. 04/ L-037)/ approved in 2011 (Annex 2) discontinued that practice. Since then the NQC decisions are final and respected by both institutions of higher education and Government institutions. Table 2: Number of evaluated HEI s per year Number of evaluated HEIs per year Year Public Sector Private Sector Total

20 Public HEIs Private HEIs Chart 4: Number of evaluated HEI s per year 2.5. Current status and the role of the Kosovo Accreditation Agency (KAA) KAA is a public agency established in 2008 by the Ministry of Education, Science and Technology (MEST) in accordance with the Law on Higher Education (No. 2002/3) (Annex 1). It is the only agency in the country responsible for the accreditation of institutions of higher education and their study programmes. KAA is independent in performing its work in accordance with the Law on Higher Education (No. 04/ L-037) (Annex 2), the Administrative Instruction on Establishing the Kosovo Accreditation Agency (No. 11/2004) (Annex 3) and the Administrative Instruction on Accreditation of Higher Education Institutions (No. 02/2013) (Annex 4). Apart from these legal acts, KAA has developed a number of internal documents which serve for the regulation of its own work as well as for the description of the accreditation procedures: the Statute of KAA (Annex 6), the Regulation of NQC (Annex 7), the KAA Principles for Programmes Offered at new/additional Locations (Annex 9), KAA Regulation on Criteria and Procedures for the Accreditation of Doctoral Studies (Annex 10), etc. KAA s primary activities concern: The accreditation of public and private institutions of higher education; The accreditation of new institutions of higher education and their programmes,(preliminary accreditation); The accreditation of new programmes at those institutions of higher education that are already accredited; Continuous control of quality at accredited institutions and their programmes; 20

21 The Kosovo Accreditation Agency mandate is: To carry out the accreditation and reaccreditation of public and private institutions of higher education and their academic and research programmes; To follow-up quality at the accredited institutions. The legal duties of Kosovo Accreditation Agency also include: Interpreting the quality requirements prescribed by law by setting guidelines and quality standards for accreditation; Developing instruments for reviewing on a regular basis whether these requirements are met by accredited institutions; Participating actively in international cooperation projects in the field of accreditation and quality assurance; Submitting an annual report on its activities to the Minister and to the Assembly. Aims: To open the sector of higher education to new providers from Kosovo and abroad; To provide quality assurance for all institutions of higher education; To promote, improve and develop the quality of the higher education sector; To create transparency and comparability in the interest of providers, students and the labour market; To encourage innovative forms and content in higher education; To ensure the comparability of degrees from Kosovo higher education institutions with those awarded by international programmes; To implement the objectives for the integration of Kosovo into the European Higher Education Area and to contribute to its development. Based on its mission and responsibilities given in the Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) (Annex 2) the principles underpinning the work of the KAA are: Its decisions shall be made in accordance with international quality standards; Its decisions shall be made with the collaboration of international experts; Its decisions shall be made independently and justified in a consistent and verifiable manner; It shall orientate itself towards educational policy developments in Kosovo and Europe; 21

22 It shall cooperate with international partners with the aim to become a member of international networks and panels in the field of quality assurance; It shall maintain communication with higher education institutions and their academic staff and students; It shall observe its accountability to the public and political decision makers by means of an effective information policy; It shall set up procedural rules, standards and evaluation criteria and make them available to the public; Its work shall rely on internationally developed good practices; It shall provide for an external evaluation of its activities on a regular basis Funding of Kosovo Accreditation Agency (KAA) KAA is funded mainly from two sources: - funds from the MEST in the form of a global allocation; KAA acknowledges that a significant proportion of resources set aside by MEST for the promotion of quality in higher education are made available thanks to the support of international donor countries; and - income generated from its activities in the form of accreditation fees and taxes paid by the higher education institutions using the Agency s services, as specified in article 23 of the Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/ 2013) (Annex 4). It should be noted that a proportion of the income generated by the Agency goes to the State budget rather than to the budget of the Agency. KAA has been maintaining a discussion with the Ministry in order to review the sharing of these resources and thus make it easier to the Agency to plan and use its budgetary resources. The annual budget is proposed by KAA based on the planned activities. The KAA budget is a separate budget line managed by MEST, which also provides administrative services for the financial management of KAA. The budget for the year 2013 is presented in Annex KAA organization and bodies Based on the Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) (Annex 2) and the Administrative Instruction on Establishment of Kosovo Accreditation Agency (No. 11/2004) (Annex 3) KAA is composed by a Board designated as the National Quality 22

23 Council (NQC) of the KAA and by a permanent Administrative structure or KAA office. The National Quality Council (NQC) is the governing body of KAA. It has 9 members appointed by the Ministry for a fixed term from among persons of both genders active in academic work in higher education and/or the professions within and outside of Kosovo. According to article 7.4 of the Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) (Annex 2) at least three members of the NQC should be international experts not employed by any higher education provider in Kosovo. The membership in NQC shall be ratified in the Assembly. All members of the Council should be expert in the field on Higher Education, having competence in evaluations and judgments, and having good communication skills. The President of the NQC is selected from its membership with open votes, for five years mandate with the opportunity to be re-elected for one more mandate. As a rule, the President should have a high academic title. The composition of the Board (the National Quality Council) has been remarkably stable since its creation, both with respect to national and international members, which has allowed it to develop its teamwork and expertise. Its current composition is as follows: Kosovo members: - Professor Ferdije Zhushi Etemi, Chair of NQC, a national expert in quality assurance and Bologna reforms in Higher education in Kosovo. One of the designers of accreditation process in the country; she is associated professor at the Department of Biology at University of Prishtina. - Professor Sabri Hamiti, one of the members with the longest experience in the NQC, member of the Academy of Science and Arts of Kosovo, writer of many books and literature parts which are used in schools, professor of Albanian language in the University of Prishtina - Professor Sadik Idriz, Bosnian nationality. Long experience in Development of Higher education policies in the country. Writer of many books of poems and short stories, professor of linguistics at the University of Prizren. Former Minister of Health in the Government of Kosovo. 23

24 - Professor Bajram Kosumi, state legislator, educator, writer and journalist. Assistant professor at Journalism Department of University of Prishtina. Former Prime Minister of the Republic of Kosovo. - Ylber Hysa, experienced independent scientific scholar in the Albanian Institute. Currently the Ambassador of the Republic of Kosovo in the Republic of Macedonia. - Doctor Isa Haxhiu, active member of Kosovo health associations and publisher of more than 20 scientific journals in Kosovo and abroad. International members: - Prof. Steve BRISTOW (UK), leading expert in quality assurance and accreditation in the UK and internationally; he specialises in strategic review, analysis and evaluation and was instrumental in drafting some of the fundamental texts on Higher Education and quality assurance in the Republic of Kosovo - Dr. Steve Crow (USA), former Chair of the Higher Learning Commission of the North Central Association of Colleges and Schools (USA); over the last decade he actively participated in global higher education quality assurance in various capacities and in different countries - Dr. Guy HAUG (Austria), one of the architects of the European Higher Education Area, with extensive experience in the evaluation of programmes, policies, higher education institutions and quality assurance agencies, mainly in Europe and Latin America. The KAA Board (NQC) has four regular meetings in a year. It takes the accreditation decision by a majority vote of its members. The Council reports annually to the Ministry and Assembly on its accreditation activities and on the adequacy of the infrastructural support available to it. The local members of NQC after being appointed by the Minister of Education a number of trainings on quality assurance at agencies for accreditation of HE in different European countries, such as Austria, Albania and UK and also in USA. Members of NQC were actively engaged in organizing seminars and trainings for HEI on the topic of Quality Assurance in higher education and the preparation of the application for accreditation. One member of NQC participates in the site visits together with the international expert team and the KAA officer, the Expert for Evaluation and Accreditation who coordinates the work of the panel. Beside the other duties they have, 24

25 NQC members participate in different international events related to QA in HE organized either by ENQA, CEENQA, EUA or other bodies. KAA s administrative office is responsible to fulfil all administrative tasks. Its main task is to organize, coordinate and supervise the accreditation process. The KAA office consists of the Executive Director of KAA and five administrative staff, who are all appointed and paid by MEST. Additionally to this, three other administrative staff members are formally planned to be employed in the next three months. The KAA Director and other staff are appointed through an open contest according to the terms and conditions of the relevant policy on recruiting staff for the civil services. The KAA Director is an ex-officio member of the National Quality Council, without right to vote. In order to manage effectively its work, KAA has developed an Internal Work Regulation of Kosova Accreditation Agency (Annex 8). Figure 1: KAA organizational scheme 25

26 According to the KAA Statute (Annex 6) the responsibilities of the KAA Director are: - Administers the process of preparing lawful and professional documentation for the agency (legal packages, standards of evaluation and accreditation, manual on procedures of evaluation and accreditation). - Coordinates the actions of KAA with MEST and NQC related to the process of evaluation, accreditation and licensing. - Plans the procedures of internal evaluation according to the published standards. - Plans the procedures of external evaluation and accreditation according to the published standards. - Organises Evaluation teams for external evaluation and recommends to NQC for approval. - Coordinates the work of evaluation teams. - Coordinates the preparation of summary reports from the recommendations of evaluation reports and delivers them to NQC for approval. - Implements all NQC decisions. Other staff working for KAA includes administrative positions which perform all administrative and legal related works in the office: - one Expert for Accreditation and Evaluation who is responsible for organizing and managing actions and processes for evaluation and accreditation of HEIs; - one Senior Legal Officer who is responsible for the overall applicability of legality within KAA, including drafting and monitoring the implementation of KAA regulations; - two Officers for Monitoring and Statistics who are in charge to mainly plan, develop, coordinate and implement inspection visits to accredited HEIs, and to maintain accurate statistical data for accredited HEIs related issues (still vacant); - one Manager for Development Projects and Quality who is mainly responsible for developing and monitoring KAA projects related to quality, accreditation, international cooperation and publications (still vacant); - one Officer for Administration and Personnel who is in charge to manage all issues related to administration, personnel, including data, records, dossiers of staff working for the KAA; 26

27 - one Officer for Budget and Finances who is in charge for managing all procedures related to finances, supplies, accounting in the KAA, including management of KAA assets; - one Officer for Publication, Web and IT who is responsible for the functioning, administration and maintenance of the ICT system in KAA KAA s international activities Since its establishment KAA has had support from a very good relationship to accreditation bodies in different countries in Europe. Thanks to this cooperation KAA came closer to existing QA networks in Europe: in 2009 KAA become member of CEEN (Network of Central and Eastern European Quality Assurance Agencies in Higher Education) now CEENQA; in 2011 KAA become affiliate to ENQA, aspiring full membership in coming years, in 2013 KAA become member of INQAAHE (International Network for Quality Assurance Agencies in Higher Education); In order to see the experiences of other QA agencies and to base its development in the best practices, KAA visited other agencies in European countries (Austria, Ireland, UK, Albania, Bulgaria) and in the USA (CHEA, APA). In order to support HEI to establish and strengthen the QA systems, KAA participated as a partner in Tempus, WUS Austria, European Council, WUS Kosovo, projects; With the aim to follow the trends in QA in HE in Europe and widen its cooperation with other QA agencies, KAA every year regularly participates in European Quality Assurance Forum, in ENQA and EUA conferences and seminars and other activities; Two members of KAA are members of ENQA working groups; In 2014 KAA is going to be the host of the CEENQA annual meeting; KAA considers the engagement of international experts in accreditation process in Kosovo as one of the strongest element of internationalization of its work. 27

28 3. KAA s ACCREDITATION ACTIVITIES Since its establishment in 2008 KAA s main activity has consisted in conducting institutional and programme accreditation procedures in higher education, in accordance with the applicable Law on Higher Education (Annex 1 and 2) and Administrative Instructions (Annex 3, 4 and 5) and in full compliance with the quality standards set out in part II of the ESGs. Accreditation is all the more important in Kosovo that it is a compulsory requisite before a private provider of higher education can be licensed and start its activities (Administrative Instruction on Licensing of Private Bearers of Higher Education in Kosova / No. 25/2012 (Annex 5). Hence, being accredited by KAA confirms that an institution fulfils the minimum quality standards to operate as an institution of higher education and can thus be licensed. KAA carries out both initial accreditations and reaccreditations. It conducts external quality assurance processes at the level of: study programmes institutions institutions and their study programmes. A summary of KAA s accreditation activities performed since 2009 at institutions of HE in Kosovo is presented in Chart 6, 7 and Accreditation procedures and decisions All accreditation procedures applied by KAA have four steps: - Step 1: Submission to the Agency of the application (Self-Evaluation Report of the institution of HE) - Step 2: External evaluation by a group of experts, including a site-visit to the institution - Step 3: Receiving accreditation from the National Quality Council and publication of the evaluation report - Step 4: Follow up procedure and improvement plan 28

29 3.2. Flow chart of accreditation procedures 29

30 Accreditation procedures start with the submission of the application, Self-Evaluation Report (SER) and First Page (FP) in English and Albanian language by the applicant higher education institution. HEIs draft the SER based on the applicable document published by KAA (KAA Guidelines on drafting the Self-Evaluation Report (SER) for evaluation of higher education institutions and their study programs (G1) (Annex 13) and Guideline on drafting the Self-Evaluation Report (SER) for program evaluation (G2) (Annex 14). After this application is checked for completeness by KAA staff and approved for evaluation by NQC, the members of the Review Panel are being selected by KAA s Director, mostly from lists of qualified evaluators proposed by KAA s Board and staff. Due to the country s size and in order to completely avoid any conflict of interest, as it is regulated by the Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) (Annex 2), all panel members, including the student member, are drawn from abroad, which means that KAA evaluation always provide the views of an international team of evaluators. A site visit to the applicant institution is a standard part of the procedure. The Review Team panel compiles an Evaluation Report based on the SER submitted by the HEI as well as on findings collected during the site visit and on further documents and evidence requested from the applicant. The first draft of the panel s Evaluation Report is sent to the institution to give its comments. The panel s final Evaluation Report contains recommendations for improvement addressed to the HEI and a recommendation for a decision addressed to the NQC. The final decision about the acceptance or rejection of the accreditation is the responsibility of KAA s Board - the NQC. The decision is taken on the basis of a summary document a Report at a Glance (RaG) compiled by the Secretariat on the basis of all information available, i.e. the Panel s report and recommendations and the HEI s observations. The RaG contains the Panel s statements on each KAA quality standard and recommendations for the further improvement of the HEI and study programs under evaluation. It serves only for the NQC members to make easier their decision on accreditation and it is never published. 30

31 Institutional and programme accreditation procedures may lead to the following decisions taken by the National Quality Council: positive decision (for accreditation) that recommends to the MEST to issue the license allowing the HEI to perform the activity; negative decision (denial of accreditation): no license will be issued by the Ministry and the HEI cannot perform the activity; conditional accreditation, if the team of experts recommends accreditation under certain conditions; in this case the NQC defines the conditions that need to be fulfilled and the deadline by which the HEI must demonstrate fulfilment; in case of a failure to fulfil the conditions, the programme or institution will not be accredited and will not get the license to operate. After the completion of the procedure the applicant institution receives from KAA the final Evaluation Report and a Notification of the accreditation decision. When the accreditation is granted, KAA issues to the HEI an Accreditation Certificate which contains information about the accredited study programmes and degrees. KAA also publishes the Evaluation Reports ( Expert Reports ) on the Agency s webpage. The accreditation decision is also being sent to the MEST for the purpose of issuing or extending the corresponding license(s), or denying the license if the accreditation answer is negative. After the accreditation procedure is accomplished, the NQC President writes an annual report for the MEST and the National Assembly. 3.3 KAA-defined standards and procedures Before it has organized the first external quality assurance procedure, KAA in compliance with ESG, developed all accreditation standards (Annex 11) and procedures and made them public through it website. In cooperation with MEST the accreditation standards, criteria and procedure were made official in an Administrative Instruction on Accreditation of Higher Education Institutions in the Republic of Kosovo, (No. 02/2013) (Annex 4), which served as a guideline for institutions. Additionally to this KAA developed KAA Guideline on drafting the Self-Evaluation Report (SER) for evaluation of higher education institutions and their study programs (G1) (Annex 13) and Guideline on drafting the Self-Evaluation Report (SER) for program evaluation (G2) (Annex 14), the lists of 31

32 requirements which applicant institutions have to demonstrate that they meet in written form. A set of documents for the use of members of evaluation panels such as KAA Guideline for experts (Institution) (Annex 16), KAA Guideline for experts (Program) (Annex 17) and KAA Code of Good Practice for Site Visits (Annex 18) was produced and sent to the selected experts. The review panel was nominated by NQC, based on the published document Selection Procedure and Criteria for Selecting Experts for the Review Teams (Annex 15), among international experts in QA based on the recommendation from partner agencies in Europe. The number of reviewers was decided based on the size of the institution and number of the study programmes. Before visiting the institution under assessment, evaluation team members signed a document the Declaration of Confidentiality (Annex 19) to declare in writing that they will carry out their tasks to the best of their ability and knowledge according to the standards of their discipline and that there are no grounds on which their impartiality could be challenged. All private barriers of HE which licenses were revoked a year before by MEST after BAC evaluation, reapplied for accreditation at both institutional and programme level. KAA received also a number of applications for new institutions. The only historical public university, the University of Prishtina, which was only formally accredited by UNMIK without any assessment, later was also the subject of external evaluation by KAA Different types of accreditation procedures External quality assurance for study programmes Since the Bologna Declaration was accepted in 2001 by the only public university that existed in the country at that time, all programmes that were subject to external evaluation once the new Agency was created were required to meet the principles of the Bologna Process (allocation of ECTS credits, two cycles of study, Learning Outcomes, etc.), both at public and private HEIs. In its first year of operation (2009), KAA evaluated all existing study programmes offered by Kosovo HEIs at that time, as well as some applications for new study programmes applying for accreditation from KAA. In total 173 study programs were evaluated in The Self-Evaluation reports of HEIs were prepared according to the Administrative Instruction on the Accreditation of Higher Education Institutions in the Republic of Kosovo 32

33 (No. 02/2013) (Annex 4) as well as Guideline on drafting the Self-Evaluation Report (SER) for program evaluation (G2) (Annex 14) which describe in detail the criteria for accreditation of a study programme, published in website of the agency. The evaluation of study programmes includes a site visit which is scheduled by the KAA Director. Only few study programmes were exceptions to this procedure. The panels of experts were selected based on the expertise and experience in higher education and quality assurance, among the list of proposed international experts by partner agencies in Europe. The selection of review members was done by the director of the agency and approved by NQC. The date of the site visit was decided in cooperation with the institution under evaluation. The Expert report was written following KAA Guidelines for experts (Program) (Annex 17), a KAA document which was sent to reviewers before the site visit. Draft report of the panel of experts was sent to the institution for comments. A final Experts Report with experts statements about the fulfilment of quality standards for study programmes was sent to NQC to decide about accreditation. In the first two years of KAA s work the accreditation decisions of NQC were approved by Government, and in the following year by the Minister of Education. Since two years, after the approval of the new Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) (Annex 2), the NQC as a decision making body of KAA has strengthened its authority and the decisions of NQC are final and need not to be further approved by any other body. Based on KAA s decisions MEST issues the licenses for study programme. KAA has conducted external evaluation of study programmes in 2008, 2009, 2010, 2011, 2012 and 2013 as a result of which a total number of 441study programmes in both public and private institution were accredited. All evaluation reports for quality of the reviewed study programmes were structured according to the KAA document KAA Guidelines for Experts (Program) (Annex 17). The reports and the NQC accreditation decisions are published on the website of the agency. 33

34 Total Number of programmes Programmes of Bachelor Level Programmes of Master Level Programmes of PhD Level Programmes of Level V Integrated programmes Table 3: Number of study programmes offered in public HEI in Kosovo in 2013 *programs which were not subject of evaluation by KAA Institution University of Prishtina University of Peja University of Prizren University of Mitrovica * * * * * * Faculty of Islamic Studies 1 1 Higher Technical Professional School, Zvecan Higher Economic Professional School, Leposavic Higher Technical Professional School, Urosevac Zvecan * * * * * * * * * * * * * * * * * * Private Bearers of Higher Education Total Agricultural sciences 02. Archi. Urb. Region Art and Design 04. Business Studies a Education, Teach. T Engineering, Techn Geography & Geology 08. Humanities 09. Linguistics and Phil Law 11. Mathematics and I... Public HEIs Mdecial Sciences 13. Natural Sciencws 14. Social Sicences Communication an Other areas of study Chart 5: Public and Private HEI s programmes according to the Erasmus Subject Area Codes (ESAC) Private HEIs 34

35 Total applications Positive Decisions Negative Decisions Chart 6: Number of evaluated programmes and number of positive and negative decisions at the Private Sector Total applications Positive Deicisons Negative Decisions Chart 7: Number of evaluated programmes and the number of positive and negative decisions at the Public Sector 35

36 External evaluation of Institutions of Higher education In the first external evaluation process conducted in 2008/09 all existing HEI s, both public and private, were evaluated. The external evaluation was organized in line with the Law on Higher Education in Kosovo (No. 2002/3) (Annex 1); Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/2013) (Annex 4) and two documents published by KAA: KAA Guidelines for experts (Institution) (Annex 16) and KAA Guidelines for experts (Program) (Annex 17). During the five years of KAA activities a total number of 36 HEIs were subject of external evaluation of the quality. Among them 3 public universities, 23 colleges and two Higher Professional School were accredited while 8 evaluated HEIs, colleges or schools were rejected due to failure to meet the KAA quality standards. Institutional evaluation is a periodic activity of KAA. It is based on evaluation of submitted documents by the applicant institution (Self-Evaluation Report) and the findings during the site visit of an Expert team. Members of Expert panel were selected among the list of experts provided by partner agencies in Europe and region. They were composed by 2-7 members, depending on the number of proposed programmes for review. A student member, proposed by European Student Union (ESU) and trained in QA in HE was part of the Expert panel, too. A number of 181 international experts from 22 countries participated in accreditation process conducted by KAA since 2009 (Chart 11). (Several of them were more than once engaged by KAA within one year of the accreditation process/ Table 4 and Chart 10). Draft report of the panel of experts was sent to institution for comments. A final Experts Report with experts statement about the fulfilment of quality standards and a recommendation for accreditation was sent to NQC to decide about accreditation. Based on NQC recommendation, MEST issued or denied the licenses for operation as a higher education institution. 36

37 Accredited Public HEIs Rejected Public HEIs Chart 8: Accredited and Rejected Public institutions per year Chart 9: Accredited and Rejected Private institutions per year * There are 11 applications for institutional accreditation that have been rejected by the Agency from It must be noted that two institutions, PBHE Professional and PBHE UNIPZ, have applied twice for accreditation and were twice rejected by the Agency. 37

38 Table 4: Number of experts engaged by KAA per year (not counted as individual experts) Number of experts per year Year Public Sector Private Sector Total Total Public HEIs Private HEIs Chart 10: Number of experts engaged by KAA in accreditation process from

39 Public HEIs Private HEIs Chart 11: KAA Number of experts as individuals involved in accreditation procedures

40 Fig. 2: Accreditation flow chart 40

41 Initial accreditation and licensing According to the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2), each private provider of HE is subject to initial accreditation before it is authorized to commence its activity. A provider of HE seeking accreditation for the first time, makes a request to KAA at least one year before the expected date to obtain accreditation. KAA s quality standards evaluated during the initial accreditation are equivalent to those of reaccreditation, except those that cannot be applied before the study programmes start. The external evaluation was organized in line with the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2), Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/2013) (Annex 4) and two documents published by KAA: KAA Guidelines for experts (Institution) (Annex 16) and KAA Guidelines for experts (Program) (Annex 17). The submitted application must contain completed documentation demonstrating that the legal criteria to perform as a HEI have been fulfilled. For this purpose an Expert panel to conduct a site visit to the new HEI is organized. On the basis of evaluation procedure the Expert team compiles a joint report which was sent to the institution under review to give its comments. A final report with experts recommendation was sent to the National Quality Council to decide on accreditation. Based on NQC decisions MEST issues or denies the licenses to perform a higher education activity. Evaluation reports and NQC decision were published in KAA website. After receiving the accreditation decision all accredited HEI submitted a Follow up plan to show the implementation of the given recommendation by review panel in order to improve their quality. In case of a negative decision by NQC or any unsatisfactory answer, HEI s can Appeal, according to the published procedures. From 2009/2013 KAA received application to carry out evaluation of 22 new HEI. As a result of these activities 16 new HEI are established, whereas 11 were rejected, (Chart 8 and 9) Reaccreditation procedures KAA, in line with the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2) and Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/2013) (Annex 4), carries out reaccreditation of HEI s in the intervals no longer than five years. The reaccreditation of a HEI within a period shorter than five years can be required by the National Quality Council (KAA s Board) or by the Ministry. The reaccreditation process requires a self-evaluation process from 41

42 the applicant institution and includes a site visit by an Expert panel. The Self-Evaluation Report produced by applicant institutions are written on the basis of an Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/2013) (Annex 4) and KAA published documents KAA G1 - Guideline on drafting the SER (Institution) (Annex 13) and KAA G2 - Guideline on drafting the SER (Program) (Annex 14). The review by the panel is focused in evaluation of the HEI documents and activities, with the focus in implementation of the recommendation given in the last evaluation process of HEI. Based on KAA published procedures, each HEI within 3 months after it is accredited has to develop a Follow up plan which shows how it is going to implement each of the recommendation given by the Review panel in the last review. In cooperation with the HEI, KAA sets the date of the site visit which can last 1, 5-2 days. Review panel drafts an Evaluation report that is sent to the HEI for the comments. The Evaluation reports were written in line with two KAA documents: KAA Guidelines for experts (Institution) (Annex 16) and KAA Guidelines for experts (Program) (Annex 17), published on the website of KAA. The final experts report with recommendation for the quality improvement is sent to the National Quality Council to make accreditation decision. As a result of this procedure the Ministry will extend the period of license for HEI or will deny it in case of a negative answer for accreditation from NQC. Evaluation reports of all HEI subject to reaccreditation were published in the KAA s website. The outcome of the last reaccreditation of HEI carried out in 2013 is reaccreditation of 21 HEIs and 206 study programmes. 52 study programmes were rejected. As a consequence rejected study programmes can not admit new students. Table 5: Number of applications per year from the private sector Private Sector New applications Accredited for the first Reaccredited Rejected time /initial acc

43 Table 6: Number of applications per year from the public sector *University of Prishtina has 16 departments. In year 2009, 5 departments were evaluated, in year departments, in departments, in year departments and in year departments were evaluated Other KAA responsibilities: Public Sector New applications Accredited for the first Reaccredited Rejected time Apart from its main responsibilities to organize the accreditation process of HE institutions in Kosovo and to constantly enhance the quality of HE, KAA has other duties as described in the Law on Higher Education in the Republic of Kosovo (No.04/ L- 037) (Annex 2): recognition of academic and professional qualifications and, in particular, acting as the competent authority for the regulation of higher education qualifications in accordance with the Law on National Qualifications; Undertaking other activities that conform to European standards and coherent for higher education that are aimed at ensuring and improving the quality and standards in higher education in Kosovo; Approving the Form of Diploma supplement; Participating in international events on QA, in networks and other forms of cooperation; KAA participates in different activities and committees of MEST that have the aim to achieve the integration of Kosovo Higher Education in the EHEA. 43

44 4. STRATEGIC DEVELOPMENT PLAN OF KAA KAA has developed the following Strategic Development Plan for the quinquennium It is deeply rooted in the specific circumstances of the young country s history, overall education policies and higher education systems. Its priorities are mainly aimed at enhancing the contribution of the agency to quality development and anchoring Kosovo s Quality Assurance system and agency into the European Higher Education Area The national context The political context: - A nation in building: little international/diplomatic experience, national democratic framework under construction, aftermaths of the war, need to build or rebuild ties to neighbouring countries, continuing challenges to the Rule of Law; - A traditional economic systems, with high unemployment, limited public resources and tensions about priority areas for spending; - Still some uncertainty about the status of the young country in the international community: status at UNO, some EU countries still do not recognize Kosovo s independence; formal adhesion to the European Cultural Convention (which is a preliminary condition for membership in the EHEA) still in progress although Kosovo has applied the principles of the Convention for several years; - Significance and potential implications of the recent Brussels Agreement; - On-going support from donor countries, including in the form of support to the Government and certain Ministries (including education). The educational context: - Recognition of education as key factor for the country s future; high percentage of young people and high pressure on educational system; high demand for education at all levels; recognized need to develop vocational/technical education at all levels; - Tradition of rather good quality in primary/secondary education - with marked weaknesses in higher education and in research; - Shortage of (qualified) teaching staff; 44

45 - Educational system still experiences severe difficulties related to differences between communities: language issues, cultural differences, historical perspectives, regional minorities; - Ministry of education: short perspective on its history and structure; political divisions and strong lobbying, on-going endeavour to harmonize educational legislation with EU; - The Kosovo Education Strategic Plan (KESP), The Higher Education context The main characteristics of Kosovo s higher education system have already been presented in Section 2. From the viewpoint of Quality Assurance the current situation and stage of development of the HE system means a number of specific challenges: - Implementation of the principles of Bologna Process (a clear commitment since 2003); - The small size of the HE system, which entails that in any particular disciplinary area (and even far beyond) members of the academic community are few and know each other, whether as good colleagues or as opponents for academic or political reasons; also difficulties with internal QA; - Legacy of neglect of Kosovo in the former system: ten years of parallel systems (an underground HE system run in parallel to the official system in order to serve Kosovo s, problems with the recognition of graduate s degrees); - The difficult integration of certain minorities and regions, still vivid resentments between certain communities; - The dominance of the University of Pristina (that was the only university until recently, operated partly underground, against the official authorities and was severely hit before and during the war and is still recovering from its wounds) in the public sector and the mushrooming of small, private HEIs of all types; - Huge caveats with regard to the understanding and acceptance of external and internal quality assurance, for want of a real quality culture in both sectors; - Not yet permitted to become a formal member of the Bologna Declaration and Bologna process in spite of adhering to it very rapidly in practice over the past decade; - Past and current support from certain key donors, in particular the EU, Austria, the UK and the US). 45

46 4.2. SWOT analysis for the Agency Strengths Standards for QA, criteria and procedures fully in compliance with ESG New degree structure already in line with Bologna Process and EHEA NQC effectively independent in decision making Involvement of international experts in evaluation panels Student participation in evaluations Generally good acceptance of KAA decisions by the HEIs Professional and committed staff Active participation and contribution in international events in QA in HE Good cooperation with a number of other QA agencies and networks Transparency and openness to public and media Accreditation process in two languages, Albanian and English The agency enjoys significant consideration and support from part of the MEST Weaknesses Limited staff resources Formal appointment of NQC members by Minister of Education Small number of training opportunities for administrative staff Insufficient working and meeting space Lack of motivation of academic staff at HEI to be more involved in QA Weak cooperation and involvement of business sector in QA processes KAA director appointed by Ministry without involvement of NQC Until now, lack of system-wide reports about quality issues in Kosovo National Qualifications Framework not yet completed Limited means for an adequate follow-up on decisions (accreditation /denial of accreditation) Ministry of Finance limitations on the amount of KAA-generated income that can be used independently Rules for computer-based data systems set by government Computer-based treatment of applications and evaluation procedures still at an embryonic stage 46

47 Opportunities HE system expanding because Kosovo has the youngest population in Europe, Become a full member of the Bologna Follow-Up Group, thereby achieving full integration of Kosovo higher education in the EHEA Improve the cross-recognition of HE qualifications in order to increase student mobility Encourage curriculum review at HEI s in order to address better the needs of the labour market Maintain pressure for HEIs to imbed quality assurance and improvement processes into their daily operations Increase the role of students in internal QA processes Increase the cooperation with employers in order to contribute to the alignment of programmes and curricula with national workforce needs Increase the cooperation with more agencies in the Southeast Europe region and in Europe at large Achieve full ENQA membership Maintain and strengthen support from Donor Countries (both with respect to funding and advice) Threats Lagging economic development of the Kosovo as a nation High unemployment rate in the country, limited capacity of labour market for highly qualified jobs Inadequate national infrastructure to support adequate learning resources (i.e. libraries, laboratories, etc.) Lack of familiarity with and low support for QA, particularly among faculties and students Limited pool of Kosovo academics with high-level degrees to provide sound academic leadership throughout the variety of HEIs Low motivation, lack of facilities, and lack of incentives for research Lack of sufficient budget for KAA, in particular should Donor Countries reduce support Long and bureaucratic procedures in financial issues Lack of some adequate bodies for HE: rectors conference, national union of students, association of private providers of higher education, professional bodies in main professional areas 47

48 Loss of the support for assuring English and Albanian documentation. Lack of adequate data management opportunities allowing KAA to meet multiple needs, including its own internal quality assurance processes 4.3. Strategic priorities for the quinquennium KAA s strategic priorities are focused on four main strands: - Consolidation of the Agency s independence - Dissemination of a culture of quality - Internationalization - Building up of resources and capacities Consolidation of the Agency s independence from government and HEIs - Consolidate current practice in new AI or Law - Continue cooperation with media in order to increase the visibility and credibility of KAA as an independent agency - Explain the importance and value-added of such independency to HEIs and general public - Participation in ENQA depends on existence of a sufficient degree of independence, but is also a guarantor of independence - Review of the Agency s governance (including Board appointment, composition and performance) in order to strengthen Dissemination of a culture of quality throughout the HE system - More emphasis on curricula defined in terms of Learning Outcomes, Skills & Competences, with a view to gear them more towards graduate employability - Support for the development/strengthening of internal QA systems throughout the HEIs in the system - Special attention to the proliferation of branch campuses and multiple employment of teachers - Special attention to need for an orderly development of doctoral studies - Special attention to the inclusion of, and cooperation with HEIs in minority regions (Mitrovica) - Extension of QA system to higher VET and regulated professions once the legal basis for this is provided - More transparency in appeal procedures in order to ensure better their acceptance in community 48

49 Internationalization - Membership in ENQA/EQAR (without waiting for full status of Kosovo as a member country of the EHEA), and latter member of ECA - More active participation of more KAA staff and experts in international projects and workshops - Bring internationalisation closer to Kosovo HEIs, in particular in the area of curricula development - Pay particular attention to the issue of foreign institutions opening a branch in Kosovo Building up of resources and capacities - Increase in number of staff of Agency already approved by Ministry; this will require more office space (some plans for this already exist) - Training programmes for current and new staff, in particular with respect to international knowledge and cooperation - Ensure the Agency s capacity to manage its staff more independently from Ministry - Internal QA system, including with respect to the time dedication of Board members - Train a select number of Kosovo professors to become qualified evaluators likely to be accepted by HEI as member of review teams including also an foreign expert - Secure payment of accreditation fees 49

50 5. KAA COMPLIANCE WITH EUROPEAN STANDARDS AND GUIDELINES 5.1. ESG part ESG 2.1. Use of internal quality assurance procedures Standard: External quality assurance procedures should take into account the effectiveness of the internal quality assurance processes described in Part 1 of the European Standards and Guidelines Guidelines: The standards for internal quality assurance contained in Part 1 provide a valuable basis for the external quality assessment process. It is important that the institutions' own internal policies and procedures are carefully evaluated in the course of external procedures, to determine the extent to which the standards are being met. If higher education institutions are to be able to demonstrate the effectiveness of their own internal quality assurance processes, and if those processes properly assure quality and standards, then external processes might be less intensive than otherwise. KAA compliance with Standard: KAA conducts an accreditation system which combines both Institutional and programme accreditation. This process is based on effectiveness of internal QA mechanisms established at HEI. The quality standards set by KAA for external quality assurance are covering all the ESG part 1. KAA documents KAA G1 - Guideline on drafting the SER (Institution) (Annex 13) and KAA G2 - Guideline on drafting the SER (Program) (Annex 14), as well as Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/2013) (Annex 4) are published and support HEIs for their internal quality assurance procedures. Conclusion: Full compliance 50

51 ESG 2.2. Development of external quality assurance processes Standard: The aims and objectives of quality assurance processes should be determined before the processes themselves are developed, by all those responsible (including higher education institutions) and should be published with a description of the procedures to be used. Guidelines: In order to ensure clarity of purpose and transparency of procedures, external quality assurance methods should be designed and developed through a process involving key stakeholders, including higher education institutions. The procedures that are finally agreed should be published and should contain explicit statements of the aims and objectives of the processes as well as a description of the procedures to be used. As external quality assurance makes demands on the institutions involved, a preliminary impact assessment should be undertaken to ensure that the procedures to be adopted are appropriate and do not interfere more than necessary with the normal work of higher education institutions. KAA compliance with Standard: Chapter 3 describes the development of KAA s external quality assurance processes and procedures. Thanks to international presence in its board, KAA has developed its procedures and processes based on the best practices in Europe and USA. Before the first external evaluation process started, KAA developed and made public on its website its procedures and processes. With the aim to better inform HEI about the standards, criteria and procedures and the self-evaluation process, KAA delivered several training seminars for HEI. Conclusion: Full compliance ESG 2.3. Criteria for decisions Standard: Any formal decisions made as a result of an external quality assurance activity should be based on explicit published criteria that are applied consistently Guidelines: Formal decisions made by quality assurance agencies have a significant impact on the institutions and programmes that are judged. In the interests of equity and reliability, decisions 51

52 should be based on published criteria and interpreted in a consistent manner. Conclusions should be based on recorded evidence and agencies should have in place ways of moderating conclusions, if necessary. KAA compliance with Standard: KAA formal decision making process refers explicitly to defined quality standards that are published and easy accessible to all KAA stakeholders and to the general public. They are implemented consistently and no complaint was received from HEIs. The decisions and the Evaluation reports are published in the website. In many cases the Expert team gives a recommendation on accreditation; however NQC members analyze all the relevant documents and form their independent conclusion on the report. Each report is subject of open discussion between the board members before the voting takes place. Conclusion: Full compliance ESG 2.4. Processes fit for purpose Standard: All external quality assurance processes should be designed specifically to ensure their fitness to achieve the aims and objectives set for them. Guidelines: Quality assurance agencies within the EHEA undertake different external processes for different purposes and in different ways. It is of the first importance that agencies should operate procedures which are fit for their own defined and published purposes. Experience has shown, however, that there are some widely-used elements of external review processes which not only help to ensure their validity, reliability and usefulness, but also provide a basis for the European dimension to quality assurance. Amongst these elements the following are particularly noteworthy: insistence that the experts undertaking the external quality assurance activity have appropriate skills and are competent to perform their task the exercise of care in the selection of experts the provision of appropriate briefing or training for experts the use of international experts participation of students 52

53 ensuring that the review procedures used are sufficient to provide adequate evidence to support the findings and conclusions reached the use of the self-evaluation/site visit/draft report/published report/follow-up model of review recognition of the importance of institutional improvement and enhancement policies as a fundamental element in the assurance of quality. KAA compliance with Standard: All external quality assurance processes carried out by KAA are designed to ensure them fitness to achieve a defined purpose or objective. KAA carries out: - Initial accreditation at institutional and programme level and - Reaccreditation at institutional and programme level Both processes include the use of the self-evaluation/site visit/draft report/published report/follow-up procedure During the initial accreditation an Expert panel composed by selected international experts, including a student member in some cases, evaluates if the minimum threshold of the criteria of quality are met for accreditation of a new HEI or a new study programme. Few weeks before the site visit takes place the selected experts receive the necessary documentation from KAA, including SER of the institution, Guidelines for Expert s report but also documents that provide information about legislation on HE in Kosovo. The reaccreditation process is more oriented to quality improvement. The focus of Experts panel is to evaluate to what extend the given recommendation in the last process where fulfilled by HEI and how it impacted the quality. Student member of the panel is an international student, proposed by the pull of ESU. Conclusion: Full compliance 53

54 ESG 2.5. Reporting Standard: Reports should be published and should be written in a style which is clear and readily accessible to its intended readership. Any decisions, commendations or recommendations contained in reports should be easy for a reader to find Guidelines: In order to ensure maximum benefit from external quality assurance processes, it is important that reports should meet the identified needs of the intended readership. Reports are sometimes intended for different readership groups and this will require careful attention to structure, content, style and tone. In general, reports should be structured to cover description, analysis (including relevant evidence), conclusions, commendations, and recommendations. There should be sufficient preliminary explanation to enable a lay reader to understand the purposes of the review, its form, and the criteria used in making decisions. Key findings, conclusions and recommendations should be easily locatable by readers. Reports should be published in a readily accessible form and there should be opportunities for readers and users of the reports (both within the relevant institution and outside it) to comment on their usefulness. KAA compliance with Standard: KAA publishes all the Evaluation reports and the Accreditation decision by the National Quality Council on its website after the process is finalized. They can be seen and read by anyone interested. The Expert reports are published in original, without any intervention by KAA. The Expert reports structure is adjusted to the structure of Self-Evaluation Report by applicant institution and two KAA documents KAA Guidelines for experts (Institution) (Annex 16) and KAA Guidelines for experts (Program) (Annex 17). Conclusion: Full compliance ESG 2.6. Follow up-procedures Standard: Quality assurance processes which contain recommendations for action or which require a subsequent action plan, should have a predetermined follow-up procedure which is implemented consistently. 54

55 Guidelines: Quality assurance is not principally about individual external scrutiny events: it should be about continuously trying to do a better job. External quality assurance does not end with the publication of the report and should include a structured follow-up procedure to ensure that recommendations are dealt with appropriately and any required action plans drawn up and implemented. This may involve further meetings with institutional or programme representatives. The objective is to ensure that areas identified for improvement are dealt with speedily and that further enhancement is encouraged KAA compliance with Standard: Follow up procedure is one of the four steps of accreditation procedure. External evaluation of an Institution of HE or a study programme always contains a set of recommendation for improvement. In the Accreditation decision issued by KAA it is required by all HEI s to submit an Improvement plan in the light of the recommendation in the Evaluation report. The plan includes the time frame, the action and the resources. However, KAA has not yet in place a mechanism to check out if there is an improvement of quality, before the next accreditation time. If accreditation is conditional, KAA sets a time limit to the institution to fulfil the given condition. The fulfilment of the condition is checked by a team set for this purpose or must be proven in written form. When accreditation decision is negative, there is no need for submission of an improvement plan. KAA is aware that the further work on Fellow up procedure has to be done to make them more clearly for the HEI s. Conclusion: Substantial compliance ESG 2.7. Periodic reviews Standard: External quality assurance of institutions and/or programmes should be undertaken on a cyclical basis. The length of the cycle and the review procedures to be used should be clearly defined and published in advance. 55

56 Guidelines: Quality assurance is not a static but a dynamic process. It should be continuous and not 'once in a lifetime'. It does not end with the first review or with the completion of the formal follow-up procedure. It has to be periodically renewed. Subsequent external reviews should take into account progress that has been made since the previous event. The process to be used in all external reviews should be clearly defined by the external quality assurance agency and its demands on institutions should not be greater than are necessary for the achievement of its objectives. KAA compliance with Standard: KAA carries out external quality assurance for HEI s and their programs periodically. The Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2) defines the length of the periodical evaluation of institutions of higher education to be a maximum of 5 years. The external evaluation of quality of study programmes is organized in 3 year cycles. Conclusion: Full compliance ESG 2.8. System-wide analyses Standard: Quality assurance agencies should produce from time to time summary reports describing and analyzing the general findings of their reviews, evaluations, assessments, etc. Guidelines: All external quality assurance agencies collect a wealth of information about individual programmes and/or institutions and this provides material for structured analyses across whole higher education systems. Such analyses can provide very useful information about developments, trends, emerging good practice and areas KAA compliance with Standard: A system-wide analysis report was published by KAA in 2009, Licensing and Accreditation under the LHE in Kosovo (Annex 23), after the first process of external evaluation of quality of existing HEI s. The last two years KAA produces Annual Reports of its work for Minister of education and the Assembly. These reports contain information about KAA activities, findings and outcomes of the processes and recommendation for enhancement of quality. 56

57 KAA plans in the future to analyze all the processes for external evaluation of quality conducted so far in order to produce a comprehensive system-wide analyses report of the HE system in Kosovo. Conclusion: partly compliant 5.2. ESG part ESG 3.1. Use of external quality assurance procedures Standard: The external quality assurance of agencies should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines. Guidelines: These statements should describe the goals and objectives of agencies' quality assurance processes, the division of labour with relevant stakeholders in higher education, especially the higher education institutions, and the cultural and historical context of their work. The statements should make clear that the external quality assurance process is a major activity of the agency and that there exists a systematic approach to achieving its goals and objectives. There should also be documentation to demonstrate how the statements are translated into a clear policy and management plan. KAA compliance with Standard: The conformity of the KAA procedures and processes for external quality assurance with ESG part II are described above, in Section 5.1. Conclusion: Full compliance ESG 3.2. Official status Standard: Agencies should be formally recognized by competent public authorities in the European Higher Education Area as agencies with responsibilities for external quality assurance and should have an established legal basis. They should comply with any requirements of the legislative jurisdictions within which they operate. 57

58 KAA compliance with Standard: KAA was established in 2008 by the Ministry of Education Science and Technology according to the Law on Higher Education in Kosovo (No. 2002/3) (Annex 1) as an independent agency responsible for assessing and promoting quality of higher education in Kosovo. The responsibilities and procedures, as well as the role of the National Quality Council as KAA s governing and decision making body are regulated by articles 7 and 15 of this law. KAA independence, as one of the precondition for ENQA membership, in taking accreditation decision is guaranteed by the law and is further fostered by the adoption of the new Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2), in All KAA s regulations and procedures are developed in full compliance with ESG and made public on its website. In 2011 KAA become affiliate to ENQA and regularly participates in meetings, conferences and other QA activities at European level. Members of KAA participate in two ENQA working groups. With all activities in external quality assessment organized in the country and affiliation to ENQA, KAA has proven full compliance with requirement of the legislative jurisdiction within which operates and it is formally recognized by competent public authorities in the European Higher Education Area as an agency with responsibilities for external quality assurance with established legal basis. Besides activities in carrying out external evaluation of quality of HEI, KAA is involved in other activities such as: participation in working groups for development of legal documents on HE, development of National Qualification framework, cooperation with Parliamentary commission for HE, participation in international conferences and workshops, etc. Conclusion: Full compliance ESG 3.3. Activities Standard: Agencies should undertake external quality assurance activities (at institutional or programme level) on a regular basis. 58

59 Guidelines: These may involve evaluation, review, audit, assessment, accreditation or other similar activities and should be part of the core functions of the agency KAA compliance with Standard: In Chapter 3 are described external quality assurance activities KAA undertakes periodically at HEI s. KAA activities rely on legal acts and its documents which are published and known to all stakeholders. According to the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2) the core functions of the agency are accreditation of HEI s in a 5 year cycle and accreditation of study programmes in 3 year cycles. In the future KAA plans to organize quality audit of HEI as a mid-term evaluation between two accreditation periods. The main goal of the accreditation process is quality enhancement. In order to follow the best practices in QA in Europe, KAA regularly reviews its documents and procedures. Conclusion: Full compliance ESG 3.4. Resources Standard: Agencies should have adequate and proportionate resources, both human and financial, to enable them to organize and run their external quality assurance process (es) in an effective and efficient manner, with appropriate provision for the development of their processes and procedures. KAA compliance with Standard: The number of accreditation processes KAA carried out since 2009, as well as the number of externally evaluated HEI s and study programmes can best demonstrate that KAA has adequate human and financial resources to organize and run external quality assurance process in an effective and efficient manner 59

60 KAA for the moment has 6 full time employees who constitute the administrative part of the agency or KAA office. It is managed by the Executive director of KAA who is responsible for planning and organizing external quality assurance process. The entire staff holds a university degree adequate to the function they have. Each of the staff performs an activity that directly affects the effectiveness and efficiency of the external quality assurance process. The profile of the staff to be hired is determined by the KAA director whereas the selection process is managed by MEST who appoints a committee to interview the applicants and select one of them based on the published criteria. The KAA director is always member of this committee. The KAA office gives an important role to the staff trainings which are intended capacity building and professionalism at work. Most of the staff underwent training in the country or abroad in the field he/she is working. KAA human resources used to be limited and overloaded in the first two accreditation process but due to the impact accreditation process has in the quality enhancement of HE system in the country, Ministry of finances enabled the KAA to increase the number of employees to nine. These human resources are sufficient to KAA to effectively manage the volume of work it has. With regard to financial resources, KAA is funded from the MEST and by the incomes it generates from accreditation fees and taxes HEI s pay for accreditation process. Each calendar year KAA proposes the Annual budget which is based on a previous year. The budged has to be approved by the Ministry of Finances. The agency's budget is a separate budget line in the MEST account. It is managed by the director of the agency who signs on behalf of the agency. The amount of the self-generated incomes that KAA can use is determined by the Ministry of finances. (The annual budget in 2013 can be found in the KAA financial plan, Annex 20). A big percentage of the budget is spent on fees for international members of the Review panels. This occasionally creates some cash flow difficulties: when transfers from the Ministry are made available to KAA with a delay, the Agency cannot pay its international experts in time. The money from donors has been an important source that has supported the development of an efficient system of quality assurance and capacity development of the agency. Of course, this system may become more difficult to sustain in case donor moneys to Kosovo were to be reduced in the future. This is one main reason why KAA considers the consolidation of its resources in a sustainable way is a major strategic 60

61 priority, even though the Agency does not currently experience a shortage of resources in view of its current level of activities. Conclusion: Substantial compliance ESG 3.5. Mission statement Standard: Agencies should have clear and explicit goals and objectives for their work, contained in a publicly available statement Guidelines: These statements should describe the goals and objectives of agencies' quality assurance processes, the division of labour with relevant stakeholders in higher education, especially the higher education institutions, and the cultural and historical context of their work. The statements should make clear that the external quality assurance process is a major activity of the agency and that there exists a systematic approach to achieving its goals and objectives. There should also be documentation to demonstrate how the statements are translated into a clear policy and management plan. KAA compliance with Standard: The mission of KAA is defined according to responsibilities in the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2) and made public in the web page The mission of Kosova Accreditation Agency KAA is to assess and promote the quality of higher education in Kosovo. Through professional and transparent processes of quality assessment and by other means deemed appropriate by it, which meet international best practices, KAA ensures that the standards and quality of higher education in Kosovo meet the demands and expectations of the European Network of the Association for quality assurance in higher education. The processes described in Chapter 3 give evidence that the external quality assurance process is a major activity of the agency and that there exists a systematic approach to achieving its goals and objectives. The external evaluation processes undertaken by the Kosovo accreditation agency during the period 2009/2013 contributed to the development of a culture of quality in higher education in Kosovo and continuous quality enhancement. 61

62 In line with its mission, KAA has developed its Strategy for the period which priorities are focused on four main strands: KAA s strategic priorities: - Consolidation of the Agency s independence - Dissemination of a culture of quality - Internationalization - Building up of resources and capacities The involvement of international experts in Review panels has made the Kosovo accreditation Agency to be considered as a good practice in the Western Balkans region for avoiding the conflict of interest during the external evaluation processes and simultaneously has positively influenced the internationalization of its work. In the coming years KAA ought to update its mission statement in order to further benchmark its work with best practices in QA in Europe and beyond. Conclusion: Full compliance ESG 3.6. Independence Standard: Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders Guidelines: An agency will need to demonstrate its independence through measures, such as: its operational independence from higher education institutions and governments is guaranteed in official documentation (e.g. instruments of governance or legislative acts) the definition and operation of its procedures and methods, the nomination and appointment of external experts and the determination of the outcomes of its quality assurance processes are undertaken autonomously and independently from governments, higher education institutions, and organs of political influence while relevant stakeholders in higher education, particularly students/learners, are consulted in the course of quality assurance processes, the final outcomes of the quality assurance processes remain the responsibility of the agency. 62

63 KAA compliance with Standard: The KAA independence is reflected in all activities, starting from the planning and organizing the external evaluation process, selection of members of Expert panels and taking the accreditation decisions. Independence is defined not only by the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) article 7, (Annex 2) but also by other sub-legal regulation based on which KAA has developed its standards and procedures, Administrative Instruction on Establishment of Kosovo Accreditation Agency (No.11/2004), (Annex 3), Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/2013) (Annex 4), The Statute of Kosovo Accreditation Agency (Annex 6) and The Regulation of National Quality Council (Annex 7). The decision making body of the KAA is the National Quality Council, which takes its decisions by majority votes of its members, in complete independence from HEIs and the MEST and on the basis of the Evaluation Reports received from the international expert panels. The procedures for the involvement of international experts in Review Panels are based on Law on Higher Education in the Republic of Kosovo (No.04/ L-037) article 15, (Annex 2). The criteria for the selection of the external experts were developed by KAA and are based on the best practices in Europe. These procedures are transparent and fully managed by the agency, without any interference from any stakeholder. However, although the current situation is quite good, the KAA is aware that its independence remains fragile and needs to be further consolidated as the young Republic of Kosovo consolidates its independence from donors and its democratic/political institutions. For this reason the consolidation of the Agency s independence is one of the key strategic priorities and objectives in the period Given the specific circumstances of Kosovo, full membership in ENQA and EQAR will be of tremendous importance in this consolidation process. Another important aspect is this respect will be the close ties that the Agency tries to develop not only with the higher education community (not just the HEIs) and with society as a whole; this is why KAA pays high attention to the development of direct and regular contact (though its Chairperson) with the main information media of the country. An underlying reason is that KAA feels committed to the development of a culture of quality in higher education and in society, far beyond the development of quality assurance procedures and standards. Conclusion: substantially compliant 63

64 ESG 3.7. External QA criteria and processes used Standard: The processes, criteria and procedures used by agencies should be pre-defined and publicly available. These processes will normally be expected to include: a self-assessment or equivalent procedure by the subject of the quality assurance process an external assessment by a group of experts, including, as appropriate, (a) student member(s), and site visits as decided by the agency publication of a report, including any decisions, recommendations or other formal outcomes a follow-up procedure to review actions taken by the subject of the quality assurance process in the light of any recommendations contained in the report. Guidelines: Agencies may develop and use other processes and procedures for particular purposes. Agencies should pay careful attention to their declared principles at all times, and ensure both that their requirements and processes are managed professionally and that their conclusions and decisions are reached in a consistent manner, even though the decisions are formed by groups of different people. Agencies that make formal quality assurance decisions, or conclusions which have formal consequences, should have an appeals procedure. The nature and form of the appeals procedure should be determined in the light of the constitution of each agency. KAA compliance with Standard: KAA has developed itself its processes, criteria and procedures, in full independence from government and HEIs and with support from international Board members and experts. All these documents are publicly available on the Agency s website. As described in Chapter 3, KAA accreditation procedure has four steps: - Submission of the application- a self-assessment by the subject of the quality assurance process - an external assessment by a group of experts, including a student member and a site visits at the HEI - Decision by National Quality Council and publication of the Expert s report - Follow up Improvement plan in the light of any recommendations given by Expert panel in the report. 64

65 In Chapter 3 are described the external processes of quality assurance KAA carries out: initial accreditation of HEI and study programmes and reaccreditation. All processes include a site visit of the Expert panel to the institution subject of external evaluation. Members of the panel are selected based on published criteria and among the list of international experienced evaluators who are experts in a certain discipline of studies or qualified for institutional evaluations. The Expert panel compiles a joint report on fulfilment of the quality standards defined by KAA. Based on the Experts report and other relevant documents, the National Quality Council takes its decision on accreditation. This can be a positive answer for accreditation when the minimum quality criteria are fulfilled by the HEI or programme, or a negative one, when the minimum quality criteria are not met. As a consequence of a negative decision, HEI won t be issued a license by MEST to perform HE activities. NQC members can decide on conditional accreditation if the Expert panel recommended fulfilment of a given condition before the accreditation period starts. Since the accreditation decision has a formal consequence, KAA has put in place an internal appeal procedure in addition to the legal right to Appeal administrative decisions in Court as defined by the Law on Administrative procedures Nr. 02/L-28. Pursuant to Section IX, article 126 of this law, an institution which is not satisfied with an administrative decision may file an appeal in the form of request for reconsideration to the authority that has taken the decision, within 30 days after receiving it. In addition to this general legal provision, KAA s Board has decided to set up an internal Appeals Committee which will receive and review the appeals and prepare in cooperation with the Secretariat a recommendation for approval or rejection that will be submitted to the NQC. This new Committee includes two Board members (one national, one international) that have just been appointed (Mr. Bristow who will act as Chair and Mr. Isa Haxhiu). The committee will have to develop its own regulations, bearing in mind the need to provide a reasoned answer to the various points referred to in the appeal and to respond within a short time (which will require online work and Board decisions). The work of this Committee will be of great importance, both as a necessary step in KAA s procedures and an as answer to the culture of appeal that seems to be developing in Kosovo higher education. Conclusion: substantially compliant 65

66 ESG 3.8. Accountability procedures Standard: Agencies should have in place procedures for their own accountability Guidelines: These procedures are expected to include the following: 1. A published policy for the assurance of the quality of the agency itself, made available on its website. 2. Documentation which demonstrates that: the agency's processes and results reflect its mission and goals of quality assurance the agency has in place, and enforces, a no-conflict-of-interest mechanism in the work of its external experts the agency has reliable mechanisms that ensure the quality of any activities and material produced by subcontractors, if some or all of the elements in its quality assurance procedure are subcontracted to other parties the agency has in place internal quality assurance procedures which include an internal feedback mechanism (i.e. means to collect feedback from its own staff and council/board); an internal reflection mechanism (i.e. means to react to internal and external recommendations for improvement); and an external feedback mechanism (i.e. means to collect feedback from experts and reviewed institutions for future development) in order to inform and underpin its own development and improvement. 3. A mandatory cyclical external review of the agency's activities at least once every five years. KAA compliance with Standard: As an agency for external quality assurance, KAA has in place the procedures for its own accountability. KAA has to be accountable for its work to its stakeholders (HEI, business sector, students and national authorities MEST and Government. It has developed a number of internal documents (The Statute of Kosovo Accreditation Agency (Annex 6), The Regulation of National Quality Council (Annex 7), KAA Guideline for institutional accreditation of branches (Annex 9), KAA Regulation on Criteria and procedures for the accreditation of doctoral studies (Annex 10) etc. which serves for regulation of its own work as well as to describe the policy for quality assurance of the agency. All these documents are published and available to the public. 66

67 The accreditation processes carried out by KAA and its results are consistent with the mission and responsibility of the agency: To assess and promote the quality of higher education in Kosovo. To conduct professional and transparent processes of quality assessment which meet international best practices To assure that the standards and quality of higher education in Kosovo meet the demands and expectations of the European Network of the Association for quality assurance in higher education. KAA has in place, guaranteed by the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2) policies to avoid conflict of interest in the work of its external experts: All members of the review panels are internationals The criteria for selection of experts are based on best practices in Europe. KAA Selection Procedure and Criteria for Selecting Experts (Annex 15) are published in the website of the agency. External experts act according to the published KAA Code of Good Practice for Site- Visits (Annex 18), External experts before the site visit to the institution have to sign the KAA Declaration of Confidentiality (Annex 19). The agency has in place internal quality assurance procedures which include an internal feedback mechanism by collecting feedback information on aspects of its activities and achievements from its own staff: Administrative staff and NQC members were involved in SWOT analyses and in the review of KAA documents and procedures. KAA has an internal reflection mechanism (i.e. means to react to internal and external recommendations for improvement): to analyze questionnaires, to analyze SWOT. The SWOT analysis served the KAA to set up its Strategic priorities for the quinquennium , presented in its Strategic Development Plan (Chapter 4) to compile an improvement plan based on the feedback from stakeholders, 67

68 to meet with representatives of HEI, to participate as partners in Tempus projects, etc this is the first external evaluation of KAA. It plans to undergo cyclical periodic review of its activities every five years. Analysis of the questionnaires In order to inform and underpin its own development and improvement KAA has put in place an external feedback mechanism to collect feedback from experts and reviewed institutions, academic staff, students and employers for future development). A survey with about 500 questionnaires for HEI (rectors, vice rectors, deans, vice deans, academic staff and students) and 146 questionnaires for experts (members of evaluation panels) (Sample of questionnaire, Annex 24) was conducted. Some of the results of questionnaires are presented in the charts below, chart 12, 13, 14). All results are presented on the website ( A total number of 268 participants (52%) from HEI and 68 from members of the Expert panels answered in the survey. The results of 3 questions from the questionnaire for HEIs and 3 questions from the questionnaire for experts are presented here. As it is shown in charts, the majority of answers to all the questions in both groups were very good or good, and only a very small percentage was poor or very poor. - When asked Have the KAA guidelines on drafting the documentation for application been understandable and sufficient (Checklist, standards and KAA criteria) 47% of participants answered it was very good and 30% it was good. Less than 7% of answers were poor and less than 1% very poor. - Regarding the professionalism of the KAA work, in the question Has the communication with the Office (KAA) been clear and professional? 63 % of participants answered very good, 22% good, less than 3% poor or very poor. - When the academic community was asked whether the accreditation process has contributed to the Internal Quality development at their institution, 62 % of the participants answered Very good, 25% good, and 2% very poor. 68

69 Chart 12: Answers of the HEIs in KAA questionnaire In the questionnaire for experts, answers were received from 67 experts. The selected questions to be analyzed are presented in the charts 12, 13 and 14). Based on the participants answers about 73% of experts were Very good informed about the tasks as an expert during the contacts prior to the visit with KAA; no expert answer was Very poor. When asked whether they did have sufficient opportunities to make all questions and address all important topics during the site visit, which questions derived from analyzing the application, 62 % of international experts answered Very good; no expert answer was Very poor. 94% of experts stated that they would be willing to be engaged again as an expert for KAA. 69

70 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Very Good Good Fair Poor Very Poor Have you been sufficiently informed about your tasks as an expert during your contacts prior to the visit with KAA, respectively during the preliminary conversation? As an expert, did you have sufficient opportunities to make all questions and address all important topics during the site visit, which questions derived from analyzing the application? Chart 13: Answers of experts in the KAA Questionnaire 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 94% 3% 3% Yes No Missing Chart 14: Answers from the question Would you be willing to be engaged once more as an expert for the KAA? 70

71 In general KAA concluded that the majority of the participants in survey have positive opinion about its work. A small number of participants also addressed some negative commentaries related to the problems faced during this short five year experience, which KAA will consider seriously and try to overcome in the next years. Conclusion: substantially compliant KAA AND ADDITIONAL ENQA CRITERIA ENQA Criterion 7 Accountability procedures (ESG 3.8) Agencies should have in place procedures for their own accountability Guidelines: These procedures are expected to include the following: 1. A published policy for the assurance of the quality of the agency itself, made available on its website. 2. Documentation which demonstrates that: the agency's processes and results reflect its mission and goals of quality assurance the agency has in place, and enforces, a no-conflict-of-interest mechanism in the work of its external experts the agency has reliable mechanisms that ensure the quality of any activities and material produced by subcontractors, if some or all of the elements in its quality assurance procedure are subcontracted to other parties the agency has in place internal quality assurance procedures which include an internal feedback mechanism (i.e. means to collect feedback from its own staff and council/board); an internal reflection mechanism (i.e. means to react to internal and external recommendations for improvement); and an external feedback mechanism (i.e. means to collect feedback from experts and reviewed institutions for future development) in order to inform and underpin its own development and improvement. 3. A mandatory cyclical external review of the agency's activities at least once every five years. 71

72 KAA compliance with Standard: 2.1. In order t to conduct professional and transparent processes of quality assessment which meet international best practices KAA has in place the documentations which demonstrate that its processes and results reflect its mission and goals of quality assurances: - KAA has positioned itself as a key actor for the development of the HE system in Kosovo. - KAA quality standards are fully in line with international quality standards - KAA has managed to move several steps forward the emergence of a quality culture in higher education in Kosovo - The presence of international experts in Expert teams gives an international dimension to the accreditation process in Kosovo 2.2. KAA has in place, guaranteed by the Law on Higher Education in the Republic of Kosovo (No.04/ L-037) (Annex 2) policies to avoid conflict of interest in the work of its external experts: All members of the review panels are internationals The criteria for selection of experts are based on best practices in Europe. KAA Selection Procedure and Criteria for Selecting Experts (Annex 15) are published in the website of the agency. External experts act according to the published KAA Code of Good Practice for Site- Visits (Annex 18), External experts before the site visit to the institution have to sign the KAA Declaration of Confidentiality (Annex 19 of SER) Regarding the internal quality assurance procedures which include an internal feedback mechanism by collecting feedback information on aspects of its activities and achievements from its own staff: KAA has in place the mechanisms to collect regularly the feedback from its staff, NQC members, international experts and stakeholders. The internal Quality assurance is involved and guaranteed through all published documents and procedures. KAA is under the discussion with WUS-Austria to 72

73 support the development of a software for Internal quality management system in order to improve the internal quality assurance procedures The quality assurance of its internal work of KAA is regulated and made transparent by the published document Rules of Procedures of Kosovo Accreditation Agency (Annex 8 of SER). Administrative staff and NQC members are regularly involved in the review of KAA documents and procedures. All the staff members of KAA as well as the members of NQC were involved and gave their contribution to the SWOT KAA has an internal reflection mechanism (i.e. means to react to internal and external recommendations for improvement): to analyze questionnaires, (the results of the Analyses of the Questionnaires are presented in pages of the SER) to analyze SWOT. The SWOT analysis served the KAA to set up its Strategic priorities for the quinquennium , presented in its Strategic Development Plan (Chapter 4) to compile an improvement plan based on the feedback from stakeholders (as a reaction to stakeholder opinion about previous KAA documents Checklist for institutional accreditation and Checklist for program accreditation, recently KAA has reviewed these two documents and published them in the new improved version as KAA G1 - Guidelines on drafting the SER (Institution) (Annex 13) and KAA G2 - Guideline on drafting the SER (Program) (Annex 14). KAA meets HEIs in different activities during the year KAA participates as partner in Tempus projects, etc 3. This is the first external evaluation of KAA. It plans to undergo cyclical periodic review of its activities every five years. 73

74 ENQA criterion 8: Consistency of judgments, appeals system and contribution to ENQA Aims I. The agency pays careful attention to its declared principles at all times, and ensures both that its requirements and processes are managed professionally and that its judgments and decisions are reached in a consistent manner, even if the judgments are formed by different groups of people. KAA ensures consistency in its judgments and decision-making by guaranteeing that they follow the procedures based on clearly defined and published criteria. In all external quality assessment procedures, starting from the submission and evaluation of the application, selection of the members of expert panels, initial accreditation procedures, program and institutional accreditation and reaccreditation procedures the decision is never taken by a single person. Due to the country s size and in order to completely avoid any conflict of interest, as it is regulated by the Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) (Annex 2), all panel members, including the student member, are drawn from abroad, which means that KAA evaluation always provide the views of an international independent team of evaluators. The Review Team panel compiles an Evaluation Report which is structures according to the requirements of KAA. The panel s final Evaluation Report contains recommendations for improvement addressed to the HEI and a recommendation for a decision addressed to the NQC. The final decision about the acceptance or rejection of the accreditation is the responsibility of KAA s Board - the NQC. The decision is taken on the basis of a summary document a Report at a Glance (RaG) compiled by the Secretariat on the basis of all information available, i.e. the Panel s report and recommendations and the HEI s observations. The RaG contains the Panel s statements on each KAA quality standard and recommendations for the further improvement of the HEI and study programs under evaluation. A great importance is given to the consistency between the panel s assessment report and decision by accreditation board (NQC). 74

75 KAA considers that the consistency of the accreditation decisions is ensured also by the: 1. Stability in the Board (The composition of the Board /the National Quality Council, has been remarkably stable since its creation, both with respect to national and international members, which has allowed it to develop its teamwork and expertise. 2. Presence of a member of the Secretariat at all site visits and 3. Role of the Secretariat in always presenting a summary of all information needed by the Board to make consistent decisions. Evaluation reports are always published in the KAA s website in Albanian and English and are easy accessible to all interested. II. If the Agency makes formal quality assurance decisions, or conclusions which have formal consequences, it should have an appeals procedure. The nature and form of the appeals procedure should be determined in the light of the constitution of the agency. Since the accreditation decision has a formal consequence, KAA has put in place an internal appeal procedure in addition to the legal right to Appeal administrative decisions in Court as defined by the Law on Administrative procedures Nr. 02/L-28. Pursuant to Section IX, article 126 of this law, an institution which is not satisfied with an administrative decision may file an appeal in the form of request for reconsideration to the authority that has taken the decision, within 30 days after receiving it. In addition to this general legal provision, KAA s Board has decided to set up an internal Appeals Committee which will receive and review the appeals and prepare in cooperation with the Secretariat a recommendation for approval or rejection that will be submitted to the NQC. This new Committee includes two Board members (one national, one international) that have just been appointed The committee will have to develop its own regulations, bearing in mind the need to provide a reasoned answer to the various points referred to in the appeal and to respond within a short time (which will require online work and Board decisions). The work of this Committee will be of great importance, both as a necessary step in KAA s procedures and an as answer to the culture of appeal that seems to be developing in Kosovo higher education. So far KAA has received 15 appeals against the NQC decisions. In three cases where new arguments were proven by HEIs NCQ has changed its decision in favour of institutions. 75

76 III. The agency is willing to contribute actively to the aims of ENQ Internationalization of its work was one of the goals of KAA since its establishment in 2008, and is set as one of the strategic priorities for the next four years. In this regard, KAA gives very much importance to the participation in different international events on QA in Higher education as well in becoming member of international networks on QA. In order to see the experiences of other QA agencies and to base its development in the best practices, KAA visited other agencies in Europe (Austria, Ireland, UK, Albania, and Bulgaria) and in USA (CHEA, APA, CACREP). In two cases KAA participated as observer during the accreditation processes, in Austria and UK. KAA become member of CEEN in 2009 (Network of Central and Eastern European Quality Assurance Agencies in Higher Education) now CEENQA; In 2011 KAA become affiliate to ENQA, aspiring full membership in this network, and recently, KAA become member of INQAAHE (International Network for Quality Assurance Agencies in Higher Education. KAA participates actively in surveys organized by ENQA and EQAR. The president of KAA s accreditation board (NQC) is a member of the Steering Committee of the Network of Central and Eastern European Quality Assurance Agencies in Higher Education, and gives her contribution to the work of this network. The next Steering Committee workshop and meeting of CEENQA in May 2014 will be hosted by KAA. With the aim to follow the trends in QA and widen its cooperation with other QA agencies, KAA regularly participates in European Quality Assurance Forum, in ENQA and EUA conferences and seminars and other activities. KAA also played an important role in the process of development of National Qualification Framework and participated in trainings and seminars organized by ETF, TAIEX, and other relevant bodies. 76

77 6. CONCLUSION KAA has produced this SER to review its five years of work, with the aim to achieve full membership in ENQA and to be listed in EQAR. Being aware of its role and responsibility in the society of a young country, KAA has positioned itself as a key actor for the development of the HE system in Kosovo. As an independent institution with solid expertise in assessing and promoting quality in Higher Education, KAA has managed to move several steps forward the emergence of a quality culture in higher education in Kosovo and to increase the awareness of HEIs that the quest for quality is the best way to make Kosova HE comparable with others in the EHEA and contribute to the economic and social development of the country. This report is the result of extensive joint work and regular meetings of the Board and staff members of the Working Group appointed for this purpose, complemented with significant input from the various categories of stakeholders. The self-evaluation process that led to the SER has also been a good opportunity for KAA to critically analyze its own work with the aim to identify its strength and weaknesses and the challenges to be faced in the future. KAA firmly plans to use the SWOT analyses and the outcomes of the stakeholders survey for the development of an improvement development plan based also on the forthcoming observations of the Review Panel. KAA therefore welcomes the visit of the evaluation team and is looking forward to its recommendations, which will be implemented in full awareness that they will allow the Agency to further enhance the quality of its work. 77

78 7. LIST OF ANNEXES Laws & Bylaws Annex 1 Law on Higher Education in Kosovo (No. 2002/3) Annex 2 Law on Higher Education in the Republic of Kosovo (No. 04/ L-037) Annex 3 Administrative Instruction on Establishment of Kosova Accreditation Agency (No. 11/2004) Annex 4 Administrative Instruction for Accreditation of Higher Education Institutions in the Republic of Kosovo (No. 02/2013) Annex 5 Administrative Instruction on Licensing of Private Bearers of Higher Education in Kosova (No. 25/2012) KAA general and internal administrative acts Annex 6 Annex 7 Annex 8 Annex 9 Annex 10 Annex 11 Annex 12 Annex 13 Annex 14 Annex 15 Annex 16 Annex 17 Annex 18 Annex 19 Annex 20 Statute of Kosovo Accreditation Agency Regulation of National Quality Council Rules of Procedures of Kosovo Accreditation Agency KAA Principles for Programs Offered at new/ additional Locations KAA Regulation on Criteria and procedures for the accreditation of doctoral studies KAA Standards KAA Application and procedure for accreditation KAA G1 - Guideline on drafting the SER (Institution) KAA G2 - Guideline on drafting the SER (Program) KAA Selection Procedure and Criteria for Selecting Experts KAA Guidelines for experts (Institution) KAA Guidelines for experts (Program) KAA Code of Good Practice for Site-Visits KAA Declaration of Confidentiality KAA Financial Plans Others Annex 21 BAC Kosovo Accreditation Project Report 2008 Annex 22 Strategy for Development of Higher Education in Kosovo Annex 23 S.B. Licensing and Accreditation under the LHE in Kosovo Annex 24 Sample of questionnaires Annex 25 Kosovo education Strategic Plan

79 Annex 8 Kosovo Accreditation Agency Rules of Procedures 79

80 KAA Kosovo Accreditation Agency KAA Office Rules of Procedures 80

81 Contents Archiving... 3 Form of letters... 5 Internal communication... 7 Mail deliveries... 7 Meetings... 9 Procedure documentation Procedure development Procedure assessment Different

82 Archiving Joint electronic dossiers All documents which are related to the Agency s procedures shall be archived in the joint dossier. Sent letters Each letter sent by the Agency must be archived in electronic and hard copy version. Every office official shall be responsible himself for the archiving. Received letters Received letters in electronic version shall be archived in electronic as well as hard copy version. For archiving in electronic version shall be resposnible also the officials who are tasked with the procedures in content wise, whereas officers tasked with the procedures in organizational wise in addition to the electronic dossier should also archive documents in hard copy and should also stamp them. Requests Any response on questions in electronic version shall be archived by the official into the electronic dossier. Interested parties/ presentations Requests and documents of interested parties for establishment of an institution of higher education should be archived in the electronic and hard copy version. Statistics should be developed for conversations held with interested parties and for presentations. Communication via telephone Notes should be taken in a certain file concerning important information that are received via telephone, and archived in the electronic dossier. 82

83 Type of letters Official signs (OS) OS shall be marked in the right, below the date. A R O CH D Accreditation Reaccreditation KAA Office (Director) NQC (Chairman) Different Higher Education Institutions (HEI) 01 UP (University of Prishtina) (...) 3-digit number A/01/ A Accreditation R Reaccreditation 01 HEI 24 Protocol s serial number 4-digit number A/01/3/ A Accreditation R Reaccreditation 01 HEI 3 No. of procedure (Example: study programme) 30 ordinal number 83

84 Office letters O/ O Office (Director) 1- Ordinal number Different letters D/ D Different 1 Ordinal number If a new application from the same institution is received than, principally, a new official number shall be given: 3-digit number A-2/01/ A Accreditation 2 Second application 01 HEI 23 Ordinal number Letters For attestations and forms white letters should be used. Font In order to have a uniformly representation in all documents the Font Book Antiqua, Size 11 shall be used. For letters, guidelines, reports, information sheets and Newsletters prepared forms shall exist. For issuing attestations, no forms shall be used but the letter with the KAA logo. 84

85 Internal communication s In principle, communication via s concerning the accreditation procedure shall be forwarded in CC to the other officer in order to follow the procedure (organizational or contents wise). Mail deliveries The mail book The exit shall include data concerning: the date, the addressed party, the type of delivery, notes for each attached document, the administrator. The entry shall include data concerning: the date, the sender, the subject, notes for each attached document, the administrator. Mail deliveries addressed to experts the control order Experts are asked to confirm by the receipt of documents sent by KAA with the accompanying letter. If no confirmation is received, the experts should be contacted in another way (by telephone, fax, etc.) at latest three weeks prior to the site visit. 85

86 Deliveries addressed to experts To experts selected by the KAA office (upon NQC approval) shall be sent an Invitation on behalf of the KAA through which it is informed for his/ her selection. The expert is asked to confirm his/ her willingness to be engaged in the procedure by filling out and sending via the confirmation for the participation of the expert. The following documents shall be sent to the expert upon his/ her positive reply: The Declaration of Confidentiality Principles of best practises for site visits Basic criteria of the Kosovo Accreditation Agency SER of the applicant institution and the accompanying documents Guidelines for experts in the accreditation procedure The Contract (calculation of travel expenses and reimbursement for the work done) The bank data sheet Modes of sending deliveries Deliveries outside the country shall be sent via express/ recomande mail. Attestations for accreditation shall be sent by mail/ recomande or shall be withdrawn from the KAA office officials. Questionnaire Feedback Questionnaires shall be sent as follows: - for experts: with the thank you note via mail and ; - for institutions: via after having issued the attestation/ decision for accreditation; - for students: via after having issued the attestation/ decision for accreditation. 86

87 Meetings of the KAA Office personnel The KAA office personnel shall be gathered for weekly meetings every Monday, from with the aim to coordinate the works for the forthcoming weeks (unless otherwise decided by the KAA Director). NQC Meetings Agenda/ time frame/ procedure status Additional data for the initiated procedure should be reviewed in alphabetical order after each meeting and should be divided in accreditation procedures, reaccreditation procedures and controlling procedure. In the meetings should be reported for the completed procedures between the two meetings. The check list for the meeting s documents Following documents should be sent together with the invitation for meeting: - Previous agenda - Time frame - Status of the accreditation procedures - The draft minutes of the last meeting (with attachments if needed). Together with the invitation other documents may be sent if needed: - Proposals for decisions - The discussion points for the agenda (principally) - Proposals for new/amended guidelines - Proposals for engagement of experts Following documents should be ready for any meeting: - Current procedures - Completed procedures Other documents may be prepared for meetings if needed. 87

88 Procedure Documentation Documents generally Documents should always contain the current status. Important documents for each procedure should have the following order: 01 Factsheet 02 Application documents 03 Expert s recommendation 04 Statement of the institution 05 Office proposal (for decision) 06 NQC decision Factsheet for procedure For every procedure a Procedure factsheet should be created in electronic version, wherein all actions, amendments and supplementations made during the procedure should be noted. The date of the last amendment (the document is found in the electronic dossier of the respective procedure) and the name of administrator should be marked as well. The current status shall be recorded also in hardcopy version. Communication All HEI statements/ remarks towards experts and the evaluation report, required for the procedure, should be found in written form ( ). Dossiers For each procedure a separate, electronic and hard copy version, should be created. Sub dossiers For special procedures the following sub dossiers shall be created (if required): - s (differentiated if required) - Engagement of experts (letters sent to experts, thank you notes, biographies). - Extracts from minutes - Additional documentation/ improvements sent A sample for dossiers is found in the HEI dossier. 88

89 Extracts of minutes Reports and decisions of meetings for special procedures shall be placed in dossier in chronological order as extracts from minutes, whereas in the electronic file as special documents. Procedure development Application All applications for accreditation should be submitted in two hardcopy exemplary (stapled copy, double side) in the KAA office. If needed, the application with improvements should be made available to members of the NQC, office and experts in 13 exemplary copies, of them 6 in the English language. In the improved application should be clearly marked the changes made from the first exemplary. For better reading and clarity of application for accreditation, KAA has made available the following forms, which should be attached to the written application documentation: Forms for accreditation application Forms for reaccreditation application Forms for accreditation application for study programmes Division of work Each procedure shall be supervised by two co officials, one of them dealing with the content wise while the other one in organizational wise. Decision for experts The office is authorized by the NQC (in previous meetings) to invite experts in order to implement the accreditation procedures planed for the academic year. The office shall prepare a proposal for experts. If the final decision for expert cannot be taken in the meeting due to the need for additional information, this can be done with the extraordinary procedure. But when taking the decision for engagement of a expert in the extraordinary procedure, professional principals of expert s team should be respected. In principle, the quality of experts has advantages towards the acceleration of procedures. 89

90 Institution s statement Usually the institution has one week time to send their statement concerning the nomination of experts. Deliveries addressed to experts The following documents are sent to experts after their positive reply to the Official Invitation: Declaration of confidentiality Principles of best practises for experts Basic criteria of Kosovo Accreditation Agency SER of the applicant institution and accompanying documents Guidelines for experts in the accreditation procedure Contracts (calculation of travel expenses and reimbursement for their work done) Bank data sheet In cases of accredited institutions: Factsheet of the respective HEI Site visit programme The site visit programme should be sent to participants two weeks prior to the visit. The copy for experts should contain the mobile phone numbers of expert s team assigned for the respective procedure. This information should not be included in the programme which is sent to the institution (exception: the mobile phone number of the Office coordinator). Experts should be informed about accurate data for the meeting (venue, time, telephone, availability). After the site visit After the site visit the following deadlines should be set: The deadline for additional documents to be sent by the institution, if needed The deadline for compiling the draft report (recommendation) The deadline for commentaries of the institution The deadline for drafting the final Report Deadlines should be set according to NQC meetings intervals. 90

91 The following timetable for the accreditation procedure should be made after the site visit and should be archived in the electronic dossier of procedures: Site visit Submission of additional documents (if needed) (maximum 2 weeks) Letter of recommendation Draft Evaluation Report (usually two weeks) Commentaries of the applicant institution (maximum 2 weeks) Letter of recommendation final report Drafting the proposal for decision (2 weeks prior the meeting) Until Until Until Until Until On NQC meeting Submitting the notification Drafting the letter of recommendation/ Draft Evaluation Report Experts, under the coordination of the team leader, have two weeks time to draft a letter of recommendation (if the institution is not required to send additional documents); otherwise individual choices for each procedure shall be made. Drafting the proposal for decision In order to draft the proposal for decision, based on which the NQC takes the decision for accreditation, the following procedure should be followed: 1. Assesment of the factual situation by experts (Final Report) 2. Drafting a proposal for decision by the KAA Director 3. Sending the proposal to the NQC 91

92 Informing experts about the decision The informing of experts about the NQC decision shall be made in two steps: The first step: brief information via after the NQC meeting The second step: together with a thank you note and the questionnaire for feedback is also send an extract of the minutes about the NQC decision via mail or . Supervision procedure After having received the annual reports of HEIs (deadline 30th of November) in two exemplary copies, one copy is sent to the reporter of supervision procedure. In order to administer annual reports a statement which provides information for reception of annual reports, drafting brief reports from the reporter, NQC decisions and the Feedback letter send to HEI, should be created (in the HEI dossier Annual reports). Procedure assessment Experts Feedback Questionnaires Feedback, thank you notes and extracts of minutes (about KAA decisions) shall be sent to experts after the accreditation decision is taken. Institution Feedback Questionnaires Feedback shall be sent to institutions after having issued the decision for accreditation via . Students Feedback Questionnaires Feedback are sent via to students who were present in the meeting of student s representatives with the team of experts during the accreditation procedure. Therefore during the site visit a list with the names and addresses of students should be complied. Coordinators Feedback The Coordinator and the KAA Director receive all questionnaires Feedback after the procedure has ended, in order to report in the future meeting for the procedure. 92

93 Dossier of the procedure assessment Below the Procedure evaluation (in the dossier for Internal Quality Assurance) links with Feedbacks of procedures in HEI should be created (in the dossier of relevant HEI below the (HEI Feedback ). Different HEI Factsheet Potential changes (personal, study programmes, etc.) shall be regularly updated. The update should be made always after having received the annual reports. Newsletter In regular basis, a Newsletter shall be compiled which includes current activities of KAA. This should be published in internet and should be sent to HEIs, students, media representatives, conferences and various associations of HEIs, MEST, other stakeholders. 93

94 Annex 9 KAA Principles for Programs Offered at new/ additional Locations 94

95 KAA PRINCIPLES FOR PROGRAMMES OFFERED AT NEW/ ADDITIONAL LOCATION 1. KAA-accredited HEIs wishing to offer accredited study programmes at a new or additional location must demonstrate that: They have gained institutional accreditation for that other location; if it is located in another country, the accreditation must be from a recognized accreditation body(*) in that country; They exercise full academic control over all their programmes offered there, including when a third-party organization is delivering them on the basis of a franchising, licensing or service agreement; The quality level and quality management available locally are comparable to those that led to the initial accreditation at the main campus, including with respect to staff qualification, academic services (e.g. libraries or ICT), administrative support (e.g. international relations, student services, career guidance) and physical facilities (as well as the local research context, in the case of research-based programmes); Running these programmes at the new/additional location is not a drain on the resources undergirding quality at the main campus. 2. Foreign HEIs wishing to open a branch campus in Kosovo, either directly of via a licensing, franchising or service agreement, must seek accreditation from KAA in the same way as Kosovar HEIs. KAA will only considered applications submitted by HEIs that are accredited by a recognized accreditation body (*) in their home country. (*) A recognized accreditation body is as defined by Article 3 of the Law on Higher Education in the Republic of Kosovo (No.04/L-037): Recognized accreditation body - Kosovo Accreditation Agency or any other accreditation body which, in the European Area of Higher Education, is in current membership of the European Network for Quality Assurance in Higher Education (ENQA) or (in the United States) is an accreditation body which has been recognized by the Council for Higher Education Accreditation (CHEA) or by the United States Department of Education (USDE) or from both or (in any other jurisdiction) has been accepted by the Ministry on a reciprocal basis as a properly constituted national or regional accrediting body, which is recognized and authorized by the relevant Ministry within that jurisdiction. 95

96 Annex 10 KAA Regulation on Criteria and procedures for the accreditation of doctoral studies 96

97 Criteria and procedures for the accreditation of institutions and programmes leading to the award of Doctoral degrees These Regulations are made under Article 7 of the Law on Higher Education in the Republic of Kosovo 2011 (Law No. 04/L-037) and should be read in conjunction with the Administrative Instruction on the Establishment of the Kosovo Accreditation Agency (No. 11/2004), the Administrative Instruction on Licensing of Private Bearers of Higher Education in Kosovo (No. 14/2003) and the Administrative Instruction for the Accreditation of Higher Education Institutions in Kosovo (No. 2/2009) or their successors. Article 1 Purpose 1.1 This Regulation sets out the criteria and procedures for the accreditation of doctoral study programmes at accredited higher education institutions in Kosovo. Its object is to regulate third level studies as defined by clause 2.3 of Article 4 of the LHE 2011 independent doctoral study programmes with academic and independent research-scientific character. Article 2 General requirements 2.1 Doctoral study programmes may be offered only by those institutions which are fully accredited by KAA or another recognized body (as defined in clause 1.16 of Article 3 of the LHE 2011) 2.2 Doctoral study programmes may be offered only in those areas of study in which the institution has academic staff who have published and been cited in refereed international journals and in which the institution has been accredited to offer Bachelors and Masters programmes and has successfully graduated at least two cohorts of students at Masters level. 2.3 If a Doctoral study programme is multi-disciplinary in nature requirement 3.2 will normally apply to all subject areas which contribute to the design of the programme. 2.4 If an institution wishes to make an exception based on the nature of the subject areas contributing to the programme or the especially innovative nature of the programme, then it must seek specific prior approval for this from KAA. 97

98 2.5 In those fields such as medicine and veterinary science where students achieve at least 300 ECTS points in their graduate studies and where direct progression from Bachelors level to Doctoral level is accepted as the normal requirement for professional practice such programmes will be scrutinised and authorised as part of the institutional accreditation for that Faculty or institution. 2.6 The title Doctor may not be used by an individual in Kosovo unless he or she has been awarded that title by an institution which has been accredited by KAA or another recognised accreditation body. Article 3 Accreditation by KAA 3.1 A higher education institution applying for accreditation of a Doctoral study programme must submit an application to the KAA at least one year (12 months) prior to the date when it intends to commence the programme. 3.2 The review of such an application will be undertaken on the basis of: a. The institution s Statute and Academic Regulations, including those Regulations which are specific to Doctoral programmes b. The most recent Expert Panel report and documentation supporting the Institutional Accreditation and relevant Programme Accreditation decisions by the National Quality Council of the KAA. c. Any further enquiries, including where appropriate a further site visit, necessary to confirm the adequacy of the institution s arrangements to secure the academic standard and integrity of the award of Doctorates. 3.3 The general conditions for accreditation set out in the Administrative Instruction for the Accreditation of Higher Education Institutions in Kosovo (AI no. 2/2009) shall apply to all such applications. Article 4 Specific requirements for accreditation of Doctoral Studies programmes 4.1 A University or University College that aspires to offer doctoral programmes in a particular area of studies must have at least three full-time professors or associate professors who hold Doctorates from an accredited institution and who are themselves active in research in the area of studies. 4.2 A college which is not a University or University College but wishes to offer doctoral study programmes in a particular area of studies must have at least two full-time professors or associate professors who hold Doctorates from an 98

99 accredited institution and who are themselves active in research in the subject area. In all cases, the judgment whether a professor is active in research will depend on the nature of the subject area but will normally require the publication of at least two peer-reviewed articles in international journals, international conference presentations or artistic productions per annum within that specific area of studies. 4.3 The institution must have its own Regulations for the admission, registration and supervision of doctoral study programme candidates. These must be consistent with its Statute and other academic regulations. These Regulations must be approved by KAA. 4.4 The institution s doctoral studies Regulations must specify the arrangements for supervision and examination of doctoral candidates and must demonstrate an adequate degree of externality in their examination. They must specify the exact role and duties of the doctoral supervisor(s) and any institutional structures established to oversee doctoral studies. In its Regulations, the institution must ensure that the process of supervision and evaluation of doctoral studies candidates is impartial and assessed at the proper level. 4.5 The institution s doctoral studies Regulations will specify the maximum number of doctoral candidates to be supervised by a doctoral supervisor at any one time. The institution will need to justify to KAA any maximum number higher than two candidates for each supervisor. The Regulations will also specify the arrangements for ensuring that the supervisor s workload is adjusted to allow enough time for supervision of doctoral candidates. 4.6 The institution must demonstrate that it has sufficient other resources of a suitable quality to support the work of the doctoral candidate(s) and the supervisor(s) of doctoral studies. This will include, but will not be restricted to, laboratories, workshops and other workspace appropriate to the needs of the candidate(s) and supervisor(s), library and IT resources and budgetary support for participation at international conferences related to the area of studies of the candidate and the supervisor. 4.7 The institution must demonstrate that it has robust quality assurance procedures to ensure authenticity of doctoral studies and research work and, in particular, to ensure comparability of doctoral awards with those of other accredited higher education institutions within the EHEA. 4.8 The institution must satisfy KAA that a research culture exists and is being supported within the institution. The institution must provide a Research Plan for each doctoral studies programme which extends for at least one year beyond 99

100 the planned duration of doctoral studies in the proposed area of study. This plan must be supported by a financial plan which demonstrates that sufficient resources will be available to implement the programme successfully. The institution must also show how the proposed programme will contribute to the economic and social development of the country. 4.9 Except in the circumstances provided for in Article 2.5 (above), no candidate shall be admitted to a Doctoral programme of studies unless she or he holds a Masters degree in that area of studies Doctoral studies Regulations shall specify the requirements for knowledge of at least one foreign language (English, German or French). This must be proven by the presentation of relevant and valid certificates showing that the candidate is able to access and understand contemporary research literature in that language within his/her proposed area of study and participate in conferences, symposia and other such international meetings Doctoral studies Regulations must also specify the language in which the doctoral thesis shall be written, examined and published. Article 5 Submitting the Application for Accreditation 5.1 The application for accreditation of a doctoral studies programme must be submitted to the Kosovo Accreditation Agency. 5.2 An application for accreditation may be submitted at any time to the KAA. 5.3 The application must be set out in such a way as to demonstrate that it meets the criteria specified in Article 4 above. 5.4 The procedure for assessing the application will be essentially the same as that set out in the Administrative Instruction for the Accreditation of Higher Education Institutions in Kosovo. 5.5 KAA has additional requirements for the selection and appointment of Experts to review applications for doctoral studies programmes. As well as being distinguished in specific branches of education and/or the professions and having experience in the processes of quality assessment and accreditation, they must also meet the following criteria: a. Possession of suitable higher level qualifications (normally a Doctorate) and knowledge of university or professional teaching in the given field. b. Research in the given field and knowledge of university research activities in their subject field. c. Experience of Doctoral supervision and examination in their subject field. 100

101 d. Experience of quality management and quality assurance inside a university. e. Independence from the institution to be evaluated and no potential conflicts of interest. 101

102 Annex 19 KAA Declaration of Confidentiality 102

103 Declaration of Confidentiality This is to declare, acknowledge and understand my responsibility as an expert for the accreditation procedure at.. in the period from... /... month /year. I hereby declare and commit to the following: Within my function as expert I agree to adhere to full confidentiality vis-à-vis third parties about all aspect of the content of the application. Furthermore I will hold in strict confidence all information, findings assessment results obtained during the site visit. Name: Signature: Data: 103

104 Annex 20 KAA Financial Plans 104

105 Budget of Kosovo Accreditation Agency The budget of Kosovo Accreditation Agency from year has been within the framework of Central Administration of the Ministry of Education, Science and Technology. In 2012 a new budgetary code for the KAA was established. In this budget were allocated funds from the state budget and own revenues of the Agency. KAA generates two types of own revenues as based on the Administrative Instruction No. 02/ 2013 on the Accreditation of Higher Education Institutions in the Republic of Kosovo dated , article 23, one part for assessment of applications and team of experts, and the other part for the student taxes. Please see below the KAA own revenues for the period : Tab. No. 1: KAA own revenues Year Amount Total , , , , , , , , , ,255 Total: 2,226,705 Chart. No.2 600, , , , , , , , , , ,000 - Buxheti 2009 Buxheti 2010 Buxheti 2011 Buxheti 2012 Buxheti

106 Buxheti ,055 Buxheti ,455 Buxheti ,055 Buxheti ,055 Buxheti , , , , , , ,000 Chart. No Të punsuarit 2009 Të punsuarit 2010 Të punsuarit 2011 Të punsuarit Të punsuarit 2013 Chart. No

107 Below are presented the KAA budget of previous years as well as the planned budget for the next year: Year Financing sources Employees Salaries and per diem rates Goods and services Utility expenses Total budget Total 4 35, , , , Governmental grants 4 25, , , , Own revenues - 10, , , Total 4 35, , , , Governmental grants 4 25, , , , Own revenues - 10, , , Total 4 35, , , , Governmental grants 4 25, , , , Own revenues - 10, , , Total 4 35, , , , Governmental grants 4 25, , , , Own revenues - 10, , , Total 9 62, , , , Governmental grants 9 53, , , , Own revenues - 9, , ,

108 Annex 24 Sample of questioners 108

109 KAA Questionnaire for experts Kosovo Accreditation Agency (KAA) is undergoing the process of internal evaluation and it has therefore compiled the following questionnaire wherein questions related to accreditation process and site visits are presented. Your contribution given through this questionnaire will serve to KAA as an instrument for improving the quality of accreditation and evaluation procedures as well as for increasing the efficiency of KAA s work. Name and last name of the expert: Institution: Procedure: Date of the site visit: The KAA representative as coordinator of the process: 109

110 1. Organisation 1.1. KAA office has contacted you at the beginning of the procedure. The first information about the procedure has been: insufficient adequate comprehensive Comments / proposals for improvement 1.2. Did you receive the documents on time in order to be able to prepare yourself for the site visit? on time not on time 1.3. Were the received documents clear and complete? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 1.4. Have you sufficiently been supported from the Office to plan your visit? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 1.5. Were you satisfied with your accommodation at the hotel? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor)

111 Comments / proposals for improvement 2. Cooperation and communication 2.1. Was the communication between you as an expert and the KAA, prior, during and after the site visit, clear and professional? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 2.2. Was the communication with other experts during the site visit sufficient and satisfactory? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 2.3. Did the preliminary conversation (the working dinner) have any impact in achieving a measurable evaluation of the application and in identifying clearly the weak points and any potential deficits? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 111

112 2.4. Have you been sufficiently informed about your tasks as an expert during your contacts prior to the visit with KAA, respectively during the preliminary conversation? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 112

113 3. Engagement of experts 3.1. Did the Guidelines for Experts provided by KAA serve as sufficient and useful assistance for the expertise? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 3.2. During the preliminary conversation, were the flow and content of the site visit addressed sufficiently and adequately? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 3.3. According to your assessment, was the composition of the expert s team well balanced? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments / proposals for improvement 3.4. As an expert, did you have sufficient opportunities to make all questions and address all important topics during the site visit, which questions derived from analyzing the application? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor)

114 Comments / proposals for improvement 4. The common impression Would you be willing to be engaged once more as an expert for the KAA? yes no Comments / proposals for improvement 5. Proposals for improvement 114

115 QUESTIONNAIRE for institutions of higher education Kosovo Accreditation Agency (KAA) is undergoing the process of internal evaluation and it has therefore compiled the following questionnaire wherein questions related to accreditation process and site visits are presented. Your contribution given through this questionnaire will serve to KAA as an instrument for improving the quality of accreditation and evaluation procedures as well as for increasing the efficiency of KAA s work. Name of the institution: Procedure: Date of the site visit: KAA representative as the process coordinator: 115

116 1. Application 1.1. Preparation of application Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) First information concerning application and procedures you get through the: a) Web page of the Kosovo Accreditation Agency (KAA) b) Office Comment/ Proposal for improvements 1.2. Have the KAA Guidelines for drafting/ preparing the application documentation been understandable and sufficient? (Check lists, standards and KAA criteria)? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comment/ Proposal for improvements 1.3. Has the request for improvements and additional documents been clear and understandable, respectively were the questions addressed concerning this point satisfactorily explained? If there has been any request for improvement please move to the next question Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comment/ Proposal for improvements 116

117 2. The process of expertise 2.1. Site-visit Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) a) Organization, time coordination and site visit program prepared by the Office b) Supervision of the site visit by the KAA representatives c) Composition of the expert s team d) Time structure of the site visit e) Topics and questions addressed Comments/ Proposal for improvement General impression about the visit (please mark below): Did the evaluation process in general (site visit, written expertise, statement on the expertise) provide sufficient opportunities to present all addressed areas in the institution? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments/ Proposals for improvement 117

118 3. Procedure flow and communication 3.1. Has the communication with the Office (KAA) been clear and professional? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Has the procedure flow, during the procedure, been sufficiently predictable and understandable to you? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Comments/ Proposal for improvements 4. Quality developments 4.1. How do you assess the contribution of accreditation procedure to the internal quality development at your institution? Please provide your evaluation for this question on a scale of 1 (very good) to 5 (very poor) Please specify three elements of the procedure which you think are particularly useful for the process of quality development at your institution General impressions 5.1. According to your assessment, which are three major weaknesses of the procedure?

119 5.2. According to your assessment, which are three strongest points of the procedure? Comments/ Proposal for improvements 119