Accreditation, evaluation and financing of universities: selected issues

Transcription

1 Accreditation, evaluation and financing of universities: selected issues Author: Juraj Nemec, Prof. PhD Affiliation: Matej Bel University Banska Bystrica, Slovakia Introduction In our paper we want to discuss some aspects of relations between university accreditation, evaluation and finance. In all developed countries, the university education is by large percentage supported by the government. The proportion between public and private financing (student fees) differs significantly between countries and normally also between different types of academic institutions in the same country. Following this practice (not testing its appropriateness) some important questions connected with public financing of universities occur, especially: 1. Which schools to finance from public funds? 2. Equal or different public grant to schools? 3. How to link performance of a school and the level of allocated grant in a most fair way? Certainly, we cannot answer such questions in definite way, but let us to try to provide some opinion and to put some open questions. 1. Accreditation and evaluation on university level In our paper we understand accreditation of the university (university programme) as the process with the final end in the form of yes or no. These yes or no may be answer to many relatively different questions, namely: Can government (its regulatory body, or any outsourced institution) include the school (programme) into nationally recognized network (normally this means to grant the right to deliver respective programme)? Can the school (programme) be accepted as the member of some national or more frequently international network as it delivers good quality? Answers to such questions should be dominantly based on careful evaluation of qualitative and normally also quantitative data describing the school of the programme. Concerning structures, approaches, tools and methods used, we might distinguish between many accreditation forms, following different characteristics, like: A: Domain : - National accreditation, valid for one country (now in EU normally for all member countries) - International accreditation 1

2 B: Accreditation body: - Accreditation completely delivered by respective state body (mainly Ministry of Education and its branches) - Accreditation awarded by the state body (Ministry), but realized by semiindependent public body (Accreditations committee, nominated by the state) - Accreditation awarded by the state body (Ministry), but realized by fully independent, normally private body (outsourcing to private accreditation bodies) C: Ways of evaluation: - Accreditation on the base of fixed set of quantitative criteria (probity) - Accreditation on the more flexible base for example on the base of set of broader standards or (and) mission and its fulfillment D: Evaluation staff: - Accreditation process realized by bosses/controllers (persons who are one level higher that evaluated staff) - Accreditation process realized by peers (colleagues, who come to check the situation to be able to help by advises how to improve) - Accreditation process realized by independent individuals, who just come to deliver professional piece of work, and do not have any specific relation to the institution E: Main goals: - To award the right to deliver a degree - To help to improve a quality of the school/programme F: Scope - Accreditation of the whole body (university or faculty) - Accreditation of partial programme Evaluation in our paper means measuring the level of performance of the university or its parts, normally by a set of quantitative (output) data (like number of scientific articles, more rarely number of graduates successfully finding job, number of visiting lectures, etc.). Evaluation is normally used for ranking of schools or their parts and frequently also as the tool for performance financing (for example in Slovakia the basic variables to set the level of the state support to the public university are number of students and evaluation parameter). Accreditation and evaluation frequently use the same or very similar indicators and measures, but serve to different purposes. 2. Accreditation effective tool to say yes to public financing? Especially CEE countries use accreditation as the tool to select universities eligible for public grants/finance. The general rule is that only (nationally?!) accredited programmes can be supported from public funds; in most countries non-accredited programmes shall not be delivered. Let us to start to think about this idea In spite, it might be taken as obvious, we still can put in front the question Do we really need accreditation? Why should anybody have the right or privilege to say yes or no to some school or its programme? 2

3 Especially liberals (ultraliberals), like J. Kinkor (1996) in the Czech Republic might argue that students (supported by parents) are best judges of the quality of a school where they want to study. And because of this there is no need for the government to establish the national accreditation system, to check if the school/programme has the right to deliver a university degree. Simple individual student s choice is preferred by right wing also because of high level of a risk connected with government failure (see for example Stiglitz, 1997). Government failure problem in connection with accreditation means that none accreditation system can be fully perfect, as it is very difficult exactly to measure a quality in fully independent way and by fully independent people. On the other hand (switching to centre and left ) can we fully accept the assumption that students really are best judges? It is relatively simple to prove the fact that majority of students follow certain strategies that prevent socially optimal choice. With imperfect labor markets the typical student s strategy might be short-term strategy, prioritizing the title/grade to quality of education received (choosing the simplest way to finish their studies). Another problem, very significant in CEE region are regional strategies, when students, because of more factors (lack of resources, family situation, etc.) prefer to enroll the nearest available university programme in acceptable field of studies. Some more strategies could be mentioned (partner, etc.) that might be accepted as individually rational, but hardly could lead to so called social optimum concerning numbers and distributions of students (probably only in phantasm of some extreme rights voices). One more reason for accreditation in the Europe cannot be avoided. Because of globalization, of main rules of common market (especially free movement of labor), and of Bologna agreements (focusing on bridging national university education systems and institutions and trying to allow students to migrate during studies), certain need to prove that the school or programme could be internationally accepted starts to be more and more actual. It is natural that the tutors of the programme want to know, if credits obtained in other school really mean that the student was also provided by necessary level of knowledge, skills and experience. How to conclude there is no perfect answer. However, the Slovak case, described bellow, (different from others??) provides important many warnings. In over-politicized societies with high risk of corruption (see Transparency International CPI indexes for all CEE countries the state really fails. In such environment: may be students choice is most transparent choice??? But again from other perspective what about sharks entering the university education market, taking public grants and not delivering the service?? Other issue: Is non-state and independent per review accreditation possible in CEE??? 2. 1 University accreditation in Slovakia: case study about main realization problems connected with accreditation The accreditation process in Slovakia involves two levels of decision making. Main part of the process is realized via the Accreditation Committee (AC), semi-independent advisory body of the Government of Slovakia. The Accreditation Committee ( has its main decision making body and several subcommittees to evaluate study programmes. The members of the AC and its subcommittees are university professors and experts from the practice, working on voluntary basis (except of small administrative staff). The system is relative simple the University sends application to accredit selected study programme, the assessment process is allocated to one of subcommittees, the committee 3

4 investigates the programme, drafts a proposal and main body reaches final decision (there is no site visit). After AC decides, it submits all proposals to the Minister of Education, who has final right to decide on awarding the accreditation. Concerning the character, the accreditation process in Slovakia is mainly based on the set of given publicly known criteria, it is accreditation of the capacity if the school to deliver respective programme, and it is the tool to say yes or now to the programme. Non-accredited programmes cannot be delivered. The set of criteria to decide is as follows: 1. The level of scientific activities of the school and its bodies participating in the programme (minimum level is set in descriptive way, not by indicators). 2. Sufficient material and technical equipment (the minimum concrete criteria is library, other aspects are set in descriptive way, not by indicators). 3. Structure of academic staff (the requirements what is sufficient are set in only descriptive way, not by indicators). 4. Quantity of academic staff (defined in the way that the maximum number of MA and BA theses per staff member is 10, MA theses can be supervised by PhD or higher qualification). 5. The structure of committees for final state examination (the minimum requirement for the committee to be approved and two members must be internal docents or professors). 6. Person responsible for the programme (there must be a full time internal person responsible for the programme, with proven experience in the field, sufficient publications and teaching experience, bellow 65 years of age, minimum docent for BA and professor for MA programmes). 7. The contents of the programme (as minimum 3/5 of the curricula must be based on national Master Curricula, the programme shall provide necessary skills). 8. Length of studies (BA: 3-4 years, MA: 1-3 years) 9. Final theses (MA and BA levels shall include final theses of prescribed structure and length). 10. Selection of students (proper approach is set in very descriptive way, not by indicators). 11. The requirements to pass examinations and to obtain degree (proper approach is set in very descriptive way, not by indicators). 12. The quality of graduates (expectations on graduates are set in very descriptive way, not by indicators). As any public sector tool, the Slovak accreditation process has it strengths and weaknesses. Potential positive aspects are peers involvement, relative independence of the AC, existence of criteria. On the other hand, it includes many critical points, limiting its effectiveness as regulative and quality control tool in following text we highlight some of such problems. The list of used indicators, provided above, clearly shows one of problematic aspects of this kind of accreditation most of criteria are not enough transparent and can be interpreted very differently, to help or to kill. It is also questionable, if all these criteria are genuine. For example, in many cases, 63 years old professor might mean from the point of view of deliverables much less than 35 years old PhD. As accreditation is the necessary precondition to get the right to deliver a university programme in Slovakia and to receive public grants, such system of criteria, in combination with the way of selection of AC and its subcommittee members, plus with the way of setting of national standard curricula creates a lot of risks, proved by reality. 4

5 AC and its subcommittee members are professors, dominantly recruited from existing long term established universities. Professors from these schools were also responsible for setting the national curricula. As the voice of experts from practice is very marginal, such arrangements give to established universities too much influence over the process. This level of influence is normally misused to grant accreditation to their programmes without real evaluation, and to block any new entrance into the market. The blocking function of accreditation in Slovakia is very visible for example in connection with the issue of attitudes to foreign universities with their branches in Slovakia. The first attempt was not to allow them to deliver degrees without accreditation. After discovering that such regulation contradicts to EU free market principles the strategy was changed only nationally accredited schools have right for public finance grants. The outcome: by the beginning of 2006 all attempts to obtain Slovak accreditation for programmes delivered by foreign schools in Slovakia failed. In many cases the Slovak national curricula were not created on the base of modern international practice, but to mirror existing structure of established programmes, very much depending on what old cadres can offer (supply driven curricula). The same is valid for approved list of study programmes (new programmes shall always fit to some of already existing category) the outcome is non-existence of standard programmes (like public finance), and existence of mish-mash name programmes (like public economics and services) in Slovakia. The accreditation process in Slovakia is expected to serve mainly as the regulatory tool, not to allow programmes of low quality to enter the market, or to sustain in the market too long. However, because of mentioned features it does not serve to its goals very effectively, even more, it limits the chance for many positive progressive developments. It allows for too much subjectivity and includes too few international comparisons. Thanks to this; it is more the fighting tool between new and established schools in their search for public finance, and not positive public policy instrument. 3. Evaluation of university performance - effective performance financing tool? The issue of evaluation of universities is connected with second and third question in our headings. When allocating public funds to the list of accredited schools (possible alternative with voucher for university students was not evaluated in CEE region, yet) as generally accepted strategy, the relative level of public grant might be the same, might differ according to the level and type of programme, but might differ also according to performance. Current public financial management principles (see for example Wright and Nemec, 2002) suggest that careful use of performance evaluation and performance financing might be positive solution replacing subjective allocation of public funds, typical for CEE in earlier phases of transition, and sensitive to corruption. However, the same theory provides also important warnings concerning performance management. In following text we focus just on problem of indicators, as one of many issues connected with performance financing. Number of students this indicator is normally used as main basis for allocation of public funds. The use of this indicator involves many problems connected with the question How to calculate the real number of students?. If number of students at the beginning of academic year is used, schools try to accept as many students as possible (lowering entry requirements and overbooking capacities). If number of graduates is used, performance requirements might be lowered. How to calculate full time and part time students? What is the proper cost difference between Ba, Ma and PhD level? 5

6 Qualification of teachers as already indicated in previous part on accreditation: Does the degree of professor really mean better quality compared to PhD? More and more authors start to warn that accreditation and evaluation opened the route to artificial professorships by awarding this degree to teachers without internationally recognized outcomes. Number of research grants system may stimulate (like in Slovakia) to submit proposals to national schemes that are preferred, and not to compete internationally. Number of quotations the use of this indicator stimulates the exchange of quotations between authors, and only best international journals try to prohibit/limit this practice. Number of articles or conference participations the explosion of a number of small local conferences with no selection of papers and no check on contents of them is reality in all CEE countries. We might to continue, but it is not necessary for this case. There is no doubt performance measurement is not simple, and no perfect solution is available generally and also in the case of performance financing of universities. Anyhow, in this case our personal position is more straightforward (compared to accreditation) performance financing, in spite of all related problems, is better choice compared to subjective allocation or block grants alternatives. The indicators and their value can be precisized step by step in open public discussions of all involved, thus bringing them closer and closer to second best. Conclusions There are many approaches to accreditation processes; in CEE accreditation is used mainly as regulatory mechanisms to set the list of publicly supported universities. The story is very general - the public body responsible for university education claims that quality control must be done by the state as students and their parents are not best judges or value/quality provided to them by a respective programme. Following this it introduces accreditation to decide, which programmes can be delivered in the market (or which will be supported from public funds). However, in reality, as any public sector tool, this type of accreditation is connected with large potential of government failure and serves to protect hidden interests. As of today, we feel that there is no chance for effective functioning of accreditation systems in typical CEE conditions, thus a lot of doubts about this tool occur. Evaluation should serve as the public financial management tool. It should set in transparently and reliable way the level of public support to schools and/or programmes. The use of evaluation is also connected with many problems, but our opinion is that there is some real chance to solve them, if open public discussion concerning indicators and their use can be generated. Literature: 1. Allen, R. Tomassi, D.: Managing Public Expenditure. Paris, SIGMA OECD Cullis, J.-Jones, P.: Microecomomics and the Public Economy: A Defence of Leviathan. Blackwell, Cullis, J.-Jones, P.: Public Finance and Public Choice. McGraw-Hill, Hamerníková, B.-Kubátová, K.: Veřejné financie. Praha, Eurolex Bohemia, Kinkor, J.: Trh a stat. Praha Kubátová, K. Vitek, L.: Daňová teorie a politika, Praha, Grada

7 7. Lane, J. E.: New Public Management. London, Routledge Medveď, J. Nemec, J. Orviská, M. Zimková, E.: Verejné financie. Sprint, Bratislava Miller, W.L., Grodeland, A.B., and Koschechkina, T.Y.: Are the People Victims or Accomplices?, Budapest, LGI, Ochrana, F.: Veřejné projekty a veřejné zakázky. Codex Bohemia, Ochrana, F.: Veřejný sektor a efektivní rozhodování. Management Press, Praha, Ochrana, F.: Veřejná volba a řízení veřejných výdajů. Ekopress, Praha, Orviská, M. a kol.: Ekonómia a financovanie verejného sektora. Fakulta financií UMB Banska Bystrica, Pollitt, Ch. Bouckaert, G.: Public Management Reform. Oxford, Oxford University Press Pudil, P. - Vybíhal, V. - Vítek, L. - Pavel, J. a kol.: Zdanění a efektivnost. Eurolex Bohemia, Praha, Rothbard, M. N.: Ekonomie státních zásahů. Liberální institut Praha, Stiglitz, J.E.: Ekonomie veřejného sektoru. Grada, Praha Strecková, Y. Malý, I.: Veřejná ekonomie pro teorii a praxi. Computer Press, Praha, Šumpíková, Markéta (ed.): Teorie a praxe ve veřejném sektoru. Megaprint, Praha Vítek, L.: Teorie společenské volby a politické rozhodování vzájemná závislost. In: Sborník příspěvků Problémy definování a prosazování veřejného zájmu. MU Brno, Wright G. Nemec. J.: Public Management in the Central and Eastern European Transition: Concepts and Cases. Bratislava, NISPAcee