Higher Education Information Systems and the Agency of Science and Higher Education

Transcription

1 Furtherance of the Agency of Science and Higher Education in its Quality Assurance Role and the Development of a Supporting Information System Higher Education Information Systems and the Agency of Science and Higher Education Remarks and Recommendations Component 3 Key Expert 3 Dr. Edgar Frackmann Zagreb,

2 Table of Contents Table of Contents Introduction Component 3 of the Project From Information Systems for Quality Assurance to National Higher Education Information System Activities and Proposals of Key Expert 3 Supporting the Development and Use of Higher Education Information Systems The Role of AZVO as the National Higher Education Information Centre Special Investigations and Surveys of the National Centre for Higher Education The Croatian National Higher Education Information System... 7 What is the Role of the Croatian National Higher Education Information System?.7 Programme Register Student Register?... 8 Data Delivery... 9 Keys and Classifications... 9 The development of the National Higher Education System and its Relationship to ISVU... 10

3 1. Introduction At the end of the Project Furtherance of the Agency of Science and Higher Education in its Quality Assurance Role and the Development of a Supporting Information System this paper is the attempt to draw some conclusions from the perspective of Key Expert 3 and the Component 3 of the project, with regards to Higher Education Information Systems and the possible role of AZVO in this context. 2. Component 3 of the Project From Information Systems for Quality Assurance to National Higher Education Information System According to the TOR Component 3 is focusing on the development of an information system for quality assurance. If one regards the quality assurance procedures the Council and the Agency are dealing with, i.e. Accreditation, Evaluation and Audit, one realizes that in each case of programme accreditation, institutional and programme evaluation, and audit, the respective institution has to provide the review panel with self-evaluation material, i.e. a self-report with a predefined set of information. This delivery of information is undertaken just for the respective evaluation process, and in the next case of evaluation, be it at another or the same institution, this information delivery has to be done again. Thus one cannot identify an information system that is used specifically for these three types of quality assurance procedures, and one cannot say that an information system is needed dedicated only for these QA procedures. However, one should expect that as part of the institutional quality management, the information required for the self-reports needs not to be generated ad hoc, just for these reports, and again and again for each of the QA procedures, but that this information (i.e. this kind of self-knowledge of the institution) is permanently available in an institutional information system which is used for the management of the educational processes including the institution s own quality assurance of these processes and their outcomes. And indeed, the majority of the Croatian Higher Education Institutions have the so called Information System for Higher Education Institutions (ISVU) implemented which helps manage the educational processes, monitor these processes from the point of view of quality assurance, and which enables the delivery of the majority of quantitative information for the self-reports. Thus if one conceives quality assurance in higher education as these three evaluation procedures only, than there seems to be no need for another information system than the existing ISVU on institutional level, apart from more procedural supporting systems like the existing Module of the Agency for Evaluation Support (MOZVAG) and an information exchange system (Internet based QA Forum) related to quality assurance issues. If one broadens, however, the concept of quality assurance beyond these explicit evaluation procedures, incorporating the information of clients, stakeholders and the public about higher education, incorporating higher education policy and planning

4 focusing on the higher education system as a whole and beyond, than one realizes that higher education information systems on a national level are missing in Croatia, systems which should enable and facilitate the communication between higher education, the relevant public (clients and stakeholders), and higher education policy (advisors and decision makers). This (national level system) gap would exist equally if ISVU would be implemented and used by all higher education institutions in the country. Thus Key Expert 3 already in the needs analysis (which had been produced in September 2006 as the first deliverable of Key Expert 3) identified, that Component 3 should not only focus on an information system, only dedicated to serve QA processes but on a Croatian national level Higher Education Information System in general, and that Component 3 should support AZVO in its role that it is taking over in this context of being responsible for information on Croatian higher education in general. 3. Activities and Proposals of Key Expert 3 Supporting the Development and Use of Higher Education Information Systems Having both the core concept of quality assurance (external accreditation, evaluation and audit) as well as the broadened concept of quality assurance in view, Key Expert 3 - proposed and promoted the wide spread use of ISVU on institutional level for the study process management including institutional quality assurance and delivery of information for external evaluations ( needs analysis, first mission) - developed a system design for a national level higher education data base and report generator (second mission) - developed a system design for a (national) study programme data base (second mission) - developed a system design for a MOZVAG upgrading (i.e. evaluation procedure supporting system for the Agency) (second mission) - developed a design for an internet platform related to quality assurance (Quality Assurance Forum) (third mission) - provided the Agency, SRCE, the Ministry and Higher Education Institutions with some information about the (overall concepts of the) German Higher Education Information Systems in order to help identify possible equivalents in the Croatian Higher Education System.(fourth mission) - provided an overall concept for higher education information systems in Croatia (fifth mission) - communicated the system designs and the overall concept for higher education information systems to the relevant stakeholders, and had intensive discussions with AZVO for clarifications and refinement of concepts, approaches and designs (sixth and seventh mission) 4

5 - developed the system specifications for the Croatian National Higher Education Information System (CNHIS), which could be used to launch a tender for development of such a system. 4. The Role of AZVO as the National Higher Education Information Centre AZVO has many roles. What would be assigned to different organizational units in other countries, due to the size of their higher education systems or due to other traditions is combined in Croatia in one organizational unit, in AZVO. AZVO is the secretariat of the Science Council, the secretariat of the Higher Education Council, the operating agency for accreditation, the higher education quality audit agency, the ENIC/NARIC office, the national Tempus office, and it is finally the national higher education information centre. All these different tasks and roles of AZVO are represented in different departments of AZVO. The department of AZVO dealing with higher education information is the Analysis and Statistics Department. What does it mean: national higher education information centre? It is the organizational unit being responsible for collecting, maintaining and providing all information on Croatian higher education, that is needed on national level, for the public and for international contexts. The national higher education information centre should be responsible for providing the Councils with information for their policy advice, providing the Ministry with information for higher education policy making, providing the National Statistics Office with data needed for higher education related statistics, providing information for international comparative studies and statistics as soon as Croatia participates, providing the public, potential students and employers with information on higher education, providing the QA related activities of AZVO and the Council with QA related information, preparing and contributing to annual national reports on higher education, be they issued by the Council or the Rectors Conference or the Ministry, organizing information exchange for QA in higher education. There are at least two main streams or methods how to collect information: one is to receive data and information regularly by the institutions (once per semester or once per year) which the institutions should be able to generate from their study process management systems (e.g. ISVU) and their resources management systems; the other stream of collecting data is through specific investigations and surveys based on samples or case studies. Thus the higher education information centre needs to maintain a data base which contains the regularly delivered data and information, and it needs to be responsible for at least the initiative, concept, organization of surveys and investigations (while these surveys and investigations might be commissioned e.g. to higher education institutions with empirical research methods competencies). With regards to the data base, i.e. the national higher education information system, as we are in a situation, where this system does not yet exist, it is the task of the higher education information centre to make it existing, to promote its existence. This 5

6 means developing the design and specifications for such a system, achieving the political decision for this system and ensuring the financing and the technical development of the system. Also the national higher education centre would have to work on being the maintainer and operator of this system, and to ensure that it is technically maintained and that the financing of the operation of this system is guaranteed. For the AZVO Analysis and Statistics department having the role of the national higher education information center, this would mean that the actual number of staff members might be too small, once all tasks are being approached. With regards to needed competencies within this group, it might have to consist of intensive knowledge and experience in higher education, some system analyst competencies, some IT competencies, some statistics competencies, and some empirical research method competencies. It would also be useful, to have the actual tasks of the department be defined and organized as projects, with clear goals, time schedule, steps with intermediate achievements and staff members of the group having their specific tasks in each of these projects. Possible projects could be e.g. - Promotion, design and development of the National Higher Education Information System - Annual Report on Higher Education - National student feedback survey - National Alumni Survey (see below) - Contribution to EUROSTUDENT (see below). 5. Special Investigations and Surveys of the National Centre for Higher Education As mentioned above the AZVO department responsible for higher education information should not only have the task to maintain the national higher education information system but also to collect relevant information which cannot be derived from this data base and from institutions data bases, i.e. to collect information based on national surveys and investigations. The need for national surveys of course has to be identified based on the need for information for problem identification or problem solving in the national higher education system context. Just to mention some examples Key Expert 3 could imagine the following national investigations and surveys (as they are conducted in other countries): - national student feedback survey (what is expected from institutions to conduct as part of their institutional quality management system, might also be conducted on a national basis in order to allow comparative views on national higher education quality and performance) - national alumni studies (what would be needed is to know e.g. how the new two cycle study system according to Bologna is accepted on the labor market, and whether and how goals that were attached to the introduction of the Bologna study structure are being met)) 6

7 - EUROSTUDENT (international comparative data on social and economic conditions of student life in Europe) (as an example of Croatian participation in international comparative statistics: Croatia seems to be an observer so far, but might join this project in the next round). 6. The Croatian National Higher Education Information System What is the Role of the Croatian National Higher Education Information System? - The system should be the central data base with all relevant and official data and information about Croatian higher education. All data delivery from the side of the Higher Education Institutions should go through this system. From here the Statistical Office for the official statistics should be served. From here the Council for policy advice and the Ministry for policy making should receive data and information. From here international comparative statistics (in case Croatia participates) should be fed with data. - The system should be at the same time the official Study Programme Register, as being a means for Higher Education Institutions to inform potential students and employers of their graduates about their study programmes. - The system should demonstrate that the institutions have the relevant information for their own quality management, and the system should provide the relevant information for the national bodies in the context of accreditation, re-accreditation and licensing of study programmes, and institutions. If a national higher education information system would be used only for statistical and reporting purposes, by far less data details would be needed to be included into the system, than has been proposed in the actual system design and system specification. All the specifications with regards to programme structure of course are useful for students to be informed, but they normally belong to institutional study process management systems. That these data items and details are included, is due to the intention of AZVO to have a data base for quality assurance processes, e.g. for calculating needed teaching capacity, for having information on the delivery of the study programmes at the institutions etc. In case the national higher education information system would not be regarded as, so to say, a permanent data delivery from the institutions as if they were permanently involved in evaluation processes, than especially the detailed data relations defined in the system specifications and many data details would not be needed, and the whole system could become less complex. The information about the programmes however, which is thought to inform potential students, potential employers of the graduates and the public should be as detailed as possible, regardless of whether the system is used for QA processes or not. 7

8 If the national higher education information system shall be used, as mentioned above, as the permanent QA data base, than a homogeneous and consistent structuring of the study programmes would be needed. A hopefully consistent structuring of programmes, partial programmes and programme branches is being suggested in System Specifications, delivered by Key Expert 3 in his seventh mission (see especially the Curriculum Plan). Identifying programmes that can be offered at more than one institution, linking these programmes to the institution where it is offered, linking (independent) modules to possibly more than one programme, with the connected courses thus being taught to students of more than one programme, identifying partial programmes that have to be combined in one study programme in which the student is enrolled, and identifying branches of programmes which after some common modules split up into different modules which students have to take according to specializations chosen, all this has to be done in a homogeneous, consistent way, already when defining and designing the programme, last but not least to exactly calculate the teaching capacity needed. Once there is the requirement of delivering data to the national data base, it is too late to make adjustments according to data structure requirements. Unfortunately it seems that in the accreditation process the institutions have not been asked to submit their programmes according to consistent rules of structuring the programmes. Thus for fulfilling the purposes of the national data base as a QA related data base it might turn out to be still necessary to produce these consistent structures of the study programmes, maybe in the course of re-accreditation. And it also might turn out that adjustments would have to me made accordingly in ISVU. Programme Register Student Register? Key Expert 3 designed the system as to be the data base for statistics and reports on a national base, and with regards to the study programme related data to be an information base as well as possibly the official national register of study programmes. In its first version the system design did not suggest to have individual students being identified, although the data have been suggested to be delivered as individual student records. Teachers appeared in the system design as identifiable even with their names, as this is an information, potential students should have when deciding to apply for enrolment in a specific programme at a specific institution. With the data suggested in the System Design document, Key Expert 3 would not see any privacy problems occurring. The public should have the right to know about the details that have been suggested to be included in the data base, including the information on the (publicly paid) teachers in higher education. The AZVO Analysis and Statistic departments suggested to have also the students identifiable in the national higher education information system, such as the citizen no would have to be included into the students records (like it is the case in ISVU). The AZVO group certainly had in mind that the national higher education information system could play the role of a national student register. Key Expert 3 cannot judge whether such a register is needed in Croatia, and whether it is on the policy agenda to have such a register. 8

9 But with having the identifiable students in the data base, a privacy problem arises, as students performance may be traced by this system. There thus have to be clear regulations as to who has the right to deal with these data, etc. It still would be easy to modify the system specifications such as not to have the students identifiable in the national system, and to leave them identifiable in the institutional systems only (i.e. e.g. ISVU). Then also the reference to the matura data base would have to be eliminated. Data Delivery The regular delivery of data from the institutions to the national higher education information system needs to be defined and regulated: - Data have to be delivered by the institutions at fixed dates in the year, according to the data structure defined in these system specifications and according to the delivery cycle suggested here and to be decided upon. - Data delivery from the institutions should be regulated by law or other legal measures. There should exist a committee inviting the relevant stakeholders, deciding e.g. once a year about possible modifications of data delivery. - As the National Higher Education Information System should not be an information system for management and not an information management system but just an information system, data has to be sent from the institutions each year or semester completely new (including historical data e.g. for each student), such that the data base may serve immediately to generate reports. - ISVU is the main supplier of data for the national system, in case ISVU is the system that is used by the institutions for their study process management. According to what the above mentioned committee decides, which data to be delivered to the national system, it might be necessary that also ISVU or the equivalent system in use at the institutions has to be adjusted. - It would have to be decided how to deal with data delivery from private institutions. Maybe the obligation to deliver data can be attached to the licensing of the institutions. Keys and Classifications A lot of keys and classifications on national basis have to be developed in order to operate such a national level system consistently. If the keys and classifications are not identifiers of the tables of this system anyway, than reference tables with these keys have to be made available in the system. The most prominent candidates for keys on national level to be developed are - the study programmes - the modules - the institutions and their subunits (organizational units) (we suggest a hierarchical key covering all higher education institutions and their subunits) 9

10 Some of the classifications to be developed and maybe existing already, which have to be included as reference tables into the system are (see for more details in the tables described in the system specifications): - teacher category - staff category - course types - exercise types - scientific area/field/branch - classifications related to socio-economic data (see in the student tables) - purpose of expenditure - expenditure category - financial source category - space category - equipment category. Key Expert 3 identifies here another task related to the role of AZVO as the national centre for higher education information. These keys must be developed centrally and maintained centrally, and it would make sense if the unit being responsible for the national higher education information system is also responsible for the development and maintenance of these keys. The development of the National Higher Education System and its Relationship to ISVU Without any doubts, with the detailed and structured study programme information and with the identifiable students the national system comes somewhat closer to ISVU. But nevertheless, ISVU is the management system for the institutions to manage their education processes, while the national system is the data base for generating information on higher education on a national comparative level. We still suggest that there is a clear separation of ISVU and the national system, being two different data bases, with a clear transfer of data according to agreed upon dates of delivery from ISVU to the national system. ISVU is the main supplier of data for the national system, in case this is the system that is used by the institutions for their study process management. According to what the above mentioned committee decides which data to be delivered to the national system, it might be necessary that also ISVU or the equivalent system in use at the institutions has to be adjusted. For the development of the National Higher Education Information System the AZVO department having the role of the national higher education information centre would not have to do very much. Going to a contractor who would develop such a system, a system specification would have to be developed, which would not need to be very detailed. In this system specification document to be used for the tender and contract with a potential developer some information should be dealt with about: 10

11 - what is the purpose of this system - what is the required output of the system - what should be the content of the system - by whom and how should it be used/accessed. What Key Expert 3 delivered as System Design and even more what he delivered as System Specifications already goes far beyond what usually might be submitted in the context of a tender document. If we ask, in which phase of the system development we are, than one might answer that we have all information needed to launch a tender for the development of the system or to immediately go into negotiations with a potential developer, provided that the system specifications do describe what AZVO and the other relevant stakeholders and financial supporters of the national higher education information system want to have. Having the given situation with regards to Croatian higher education information systems in view, and based on his experience from other countries, Key Expert 3 thought that it would be the best solution to have the new system technically developed and maintained from within the higher education system, as is the case with ISVU (and probably also the best solution for ISVU). And having in view that the development and maintenance of ISVU is really a success story, and having also in mind that ISVU is the main supplier of data for the national system, and that ISVU might need some adjustments due to the requirements from the national system, and finally that the data warehouse software might be re-used for the national system, Key Expert 3 thought that the same team, the ISVU team could also be the developer and technical maintainer of the new national system. However Key Expert 3 learnt that the maintenance and further development of ISVU seems not to be organized properly with regards to contract, finances, decision making and organizational cooperation. Key Expert 3 urgently recommends that the Ministry tries to solve the existing problems in order to have a functioning maintenance and further development organization of ISVU guaranteed, which would primarily ensure that institutions have a satisfactorily functioning study process management system, which would also ensure that the main supplier for a national level higher education information system could deliver valid and reliable data to the national level. Key Expert 3 also learnt that the two organizations involved so far mutually into the development and maintenance of ISVU, FER and SRCE, seem not to be ready any more to cooperate with the development and maintenance of the new national level higher education information system. AZVO seems to have decided to launch a tender for the technical development and technical maintenance of the system (the latter indeed being very important for the sustainability of the system). Key Expert 3 recommends, that if indeed a third party (so to say from outside the higher education system and from outside the existing experience with ISVU) is involved in the system development and maintenance of the national level information system, than it is the more urgent that the AZVO group has more system analyst and IT competence on board. And also a solution for the further development of ISVU is even more urgent. 11

12 The new national level higher education information system definitely should be a separate system, separate from the existing institutional system ISVU, separate with regards to data base as well as software. One might however consider using the ISVU data warehouse software as the basis for developing the national higher education system. And this should be and option which the bidding developer should be asked to take into account. Also the German system ICE (contact: Peter Muessig- Trapp, [email protected]) will be available as open source software. And there might be other open source software available for the Croatian system to be developed not from scratch. Of course it is less expensive, starting from existing software than developing such a system from scratch. 12