ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 1. Implementation of the ENQA recommendations on ESG Part 2

Transcription

1 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 1 Implementation of the ENQA recommendations on ESG Part 2 ESG 2.1 Use of internal quality assurance procedures (ENQA Criterion 1) External quality assurance procedures should take into account the effectiveness of the internal quality assurance processes described in Part 1 of the European Standards and Guidelines. Recommendation None None required ESG 2.2 Development of external quality assurance processes (ENQA Criterion 1 cont.) The aims and objectives of quality assurance processes should be determined before the processes themselves are developed, by all those responsible (including higher education institutions) and should be published with a description of the procedures to be used. Substantially compliant That OAQ consider a formalisation of mechanisms for the involvement of other stakeholders, especially students (as stakeholders), in the future design and development of its procedures OAQ may wish further to consider a more structured involvement of user groups in the formative drafting stage of its operating documents. Over the past ten years, two standing forums were developed in which OAQ involves its stakeholders in the design of new procedures (when given a mandate) and when developing its existing procedures (e.g. format, instruments): the Q-Netzwerk of the Rectors Conference (CRUS) and the Scientific Advisory Board. OAQ has collaborated with the national Union of Students of Switzerland (participation in training workshops and financial support) in the preparation of students for the self-evaluation and external evaluation phases of the Quality Audits for many years.

2 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 2 Based on the two recommendations OAQ suggested to the Q-Netzwerk that student representatives are given an active responsibility in the Q-Netzwerk (so far the Q-Netzwerk consisted of representatives of each university only, see Qualitätsnetzwerk); OAQ suggested to the Swiss University Conference that the range of members of the Scientific Advisory Board is extended beyond the statutory size to include a student representative. On 28 June 2012, the Swiss University Conference elected a student member to the Scientific Advisory Board (see OAQ has revised the agenda of the Scientific Advisory Board meetings, allowing a greater involvement of the Board in discussing the design and the revision of the instruments (operating documents). As part of the preparation of a third cycle of Quality Audits started in 2013, OAQ revised the format of its Quality Audit in This work was carried out in close collaboration with the Q-Netzwerk. In particular, a working group of the Q-Netzwerk, composed of QA officers, the OAQ and a student representative, revised the quality standards. OAQ collaborated with relevant stakeholders in the development of standards and guidelines and the design of the accreditation procedures of postgraduate pharmacy education programmes. The preparation of drafts of the future accreditation guidelines implementing the new law HFKG is being carried out in working groups. Student representatives as well as representatives of any conceivable stakeholder group are involved in this work (

3 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 3 ESG 2.3 Criteria for decisions (ENQA Criterion 1 cont.) Any formal decisions made as a result of an external quality assurance activity should be based on explicit published criteria that are applied consistently. Substantially compliant The publication of all OAQ reports would be helpful on the one hand as a contribution to quality enhancement, and on the other hand as the source of external and independent assessment contributing to the formulation of the final decision. This would enhance consistency and transparency. In applying for renewal of recognition as an agency in the UAS domain to the federal authorities in September 2012, OAQ demanded an explicit authorisation to submit the full experts report to the examined unit for feedback (institution or programme). In January 2013, the request of OAQ was fully accepted. Since the beginning of 2013, OAQ submits full reports to the evaluated units for feedback. Moreover, OAQ publishes a short report along with every positive decision on accreditation on its website. OAQ further introduced a summary of its findings in the published decision (see, for example, OAQ advocates the advantages of publishing all reports and decisions as a member of the relevant working group preparing the implementation of the HFKG. According to the text under discussion, the new accreditation council will publish all positive decisions on accreditation along with a report making transparent each phase of the decision-making. The report will be examined, before publication, for issues related to data and personal protection.

4 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 4 ESG 2.4 Processes fit for purpose (ENQA Criterion 1 cont.) All external quality assurance processes should be designed specifically to ensure their fitness to achieve the aims and objectives set for them. Substantially compliant That OAQ work with panel chairs to identify how to further enhance the role of experts through an enhanced induction and training process, possibly delivered through e-learning. That OAQ strengthen the consistency of its procedures for briefing international experts on the national context, in view of the necessity to explain, clearly and in a transparent and detailed way, the complexity and historical context of the Swiss HE system. That management of the data-base of experts be improved, in particular, the currency of the database. That a more formalised system to review and document the performance of experts be developed. That OAQ consider expanding the profile of expert panels to include greater professional, business and community involvement OAQ recognises the need for action with respect to the recruitment and training of experts as well as the development of their role in OAQ s procedure. Therefore, OAQ has restructured responsibilities and appointed one staff member in charge of managing the expert database revised its existing training documents in order to improve the briefing on the national context and the role of the experts has produced a flyer explaining the Swiss HE system engaged in ECA working group WG 2 realising the project E- Train in order to further develop our training of experts has introduced a half-day pre-visit for the Quality Audits, i.e. a briefing and working session for the experts to take place about a month before the planned on-site visit has set up a managed pool of experts for the Quality Audits and for the accreditation procedures of postgraduate programmes in the field of psychology participates in a linguistic study on expert communication in order to improve external evaluation practice (interviews, reporting) ESG 2.5 Reporting (ENQA Criterion 1 cont.)

5 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 5 Reports should be published and should be written in a style which is clear and readily accessible to its intended readership. Any decisions, commendations or recommendations contained in reports should be easy for a reader to find. Substantially compliant That OAQ with its stakeholders initiate further work on present and future legal frameworks for publication of reports, in the light of the emerging new legislation That OAQ undertake a comprehensive review of the extent to which reports could be tailored to meet the needs of different stakeholder groups. See actions under ESG 2.3 OAQ participates in an ENQA research project (EQArep), financed under the EC Life Long Learning programme. The basic aim of this project is to contribute to a process that improves the usability of reports for stakeholders in higher education across Europe (see the article in the Newsletter: OAQ has revised the format of the external evaluation report for the Quality Audits and plans to produce a two-page easy to read summary for the general public. Further actions In the upcoming procedures for the accreditation of psychology programmes, the OAQ responsible will act as secretary of the experts panel. The objective is to obtain homogeneous and fit for purpose reports. ESG 2.6 Follow up-procedures (ENQA Criterion 1 cont.) Quality assurance processes which contain recommendations for action or which require a subsequent action plan, should have a predetermined follow-up procedure which is implemented consistently. That OAQ could usefully review, in consultation with its stakeholders, how the informal quality enhancement aspect of feedback from the review processes could be more securely embedded in its operations. OAQ and the Swiss Universities have agreed to document a follow up of the implementation of recommendations from the previous Quality Audit procedures in their self-evaluation report of the third Quality Audit.

6 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 6 ESG 2.7 Periodic reviews (ENQA Criterion 1 cont.) External quality assurance of institutions and/or programmes should be undertaken on a cyclical basis. The length of the cycle and the review procedures to be used should be clearly defined and published in advance. Notwithstanding the impending new legislation, OAQ in conjunction with its stakeholders should not persist indefinitely with the pause in some cycles of activity. The Quality Audit was the only cycle, which suffered a pause. Action Based on a mandate by the State Secretary, OAQ initiated a cycle of Quality Audits 2013/14; 2012 was dedicated to the preparation of the instruments. The first three procedures were started at the beginning of 2013, another wave of 7 procedures started in summer 2013, and the last two will start at the end of ESG 2.8 System-wide analysis (ENQA Criterion 1 cont.) Quality assurance agencies should produce from time to time summary reports describing and analysing the general findings of their reviews, evaluations, assessments, etc. Substantially compliant development of a proposal for an expansion of and a more systematic approach to its activities in sector-wide analysis development of an outline work plan of future activity in sector-wide analysis undertaking a skills analysis within OAQ to determine what further HR capacity is needed to support this work, for example in data management and analysis After each accreditation cycle, OAQ publishes a synthesis report that summarises the results of a critical analysis of outcomes and procedures. OAQ participates in an ENQA working group on stakeholder involvement OAQ is systematically collecting information on relevant projects in the field of QA-related Higher Education Research

7 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 7 discussing with stakeholders the way in which additional OAQ capacity can additionally help to underpin and in due course evaluate the effectiveness of new methodologies introduced through the new legislation Further actions see under ESG 3.4 (staff development) discussing with stakeholders future funding arrangements for enhancing OAQ s analytical capacity.

8 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 8 Implementation of the ENQA recommendations on ESG Part 3 ESG 3.1 Use of external quality assurance procedures for higher education (ENQA Criterion 1 cont.) The external quality assurance of agencies should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines. Substantially compliant see detailed comments and recommendations under ESG Part 2 and under section 3.3 see respective actions ESG 3.2 Official status (ENQA Criterion 2) Agencies should be formally recognised by competent public authorities in the European Higher Education Area as agencies with responsibilities for external quality assurance and should have an established legal basis. They should comply with any requirements of the legislative jurisdictions within which they operate. None No action required ESG 3.3 Activities (ENQA Criterion 1 cont.) Agencies should undertake external quality assurance activities (at institutional or programme level) on a regular basis. None No action required

9 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 9 ESG 3.4 Resources (ENQA Criterion 3) Agencies should have adequate and proportionate resources, both human and financial, to enable them to organise and run their external quality assurance process (es) in an effective and efficient manner, with appropriate provision for the development of their processes and procedures (and staff) (Addition by ENQA for ENQA criterion) That OAQ be encouraged to complete its HR strategy and to develop a more formal and complete procedure for the induction and subsequent professional development of staff OAQ is working on an HR strategy to be finished by the end of OAQ has established a schedule to allow every staff member to undertake ENQA s expert training. OAQ is taking an active part in the ENQA working group on Staff development. A staff member of the OAQ is involved in a continuous education programme on management of public administration with a focus on HR development. ESG 3.5 Mission statement (ENQA Criterion 4) Agencies should have clear and explicit goals and objectives for their work, contained in a publicly available statement. Substantially compliant That further work is required to articulate more explicit goals for the agency and to set out more clearly how the Agency will interact more proactively with other stakeholders such as professional bodies, employers and society more broadly. An internal OAQ task force has developed a new Strategy and an Action Plan for OAQ submitted preliminary drafts and the final version to the Scientific Advisory Board and the Swiss University Conference SUC. SUC endorsed the Strategy and Action Plan in its newsletter ( d=33, CUS Info No 2/2013).

10 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 10 The Strategy and Action Plan is published on the OAQ web site: Its visibility is strengthened by a flyer. In 2012, OAQ relaunched its newsletter which is broadly distributed and open to expressing the views of all stakeholders in its columns. Further actions OAQ will strengthen the involvement of the professional world and society at large (according to Objective 2 published in the OAQ strategy)oaq will discuss with SUC and the Scientific Advisory Board the issue of including a member from the professional bodies. OAQ will discuss with SUC and the Scientific Advisory Board the issue of including a member from the professional bodies. OAQ plans to organise annual events with stakeholders in order to exchange ideas on chosen topics ESG 3.6 Independence (ENQA Criterion 5) Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders. Substantially compliant For the remaining period of the current legislation, OAQ should maintain a clear tracking of any differences between the conclusions of OAQ reports and those of any higher formal decision making body (such as OPET). In the interests of transparency, both versions should be published separately, unless expressly prohibited by law, or fed back in some other way. OAQ now guarantees full transparency to evaluated institutions with respect to the recommendation on accreditation pronounced by the expert team and by the Scientific Advisory Board. In its application for renewal of the authorisation as agency active in the Swiss UAS sector, OAQ demanded to be allowed to communicate the full version of the experts' report to the UAS (including conditions and recommendations). The ministry accepted this, now guaranteeing full transparency. OAQ regularly reports on conclusions taken by formal decision making

11 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 11 ESG 3.7 External quality assurance criteria and processes used by the agencies (ENQA Criterion 6) bodies to the Scientific Advisory Board, enabling a note of any differences with the own conclusions. OAQ now keeps a Book of decisions showing all the contributions of the various bodies to the decision, most notably the development from the experts conclusion to OAQ s proposal and the final decision by the deciding body. The processes, criteria and procedures used by agencies should be pre-defined and publicly available. These processes will normally be expected to include: a self-assessment or equivalent procedure by the subject of the quality assurance process an external assessment by a group of experts, including, as appropriate, student member(s), and site visits as decided by the agency publication of a report, including any decisions, recommendations or other formal outcomes a follow-up procedure to review actions taken by the subject of the quality assurance process in the light of any recommendations contained in the report. Substantially compliant That OAQ re-write its documentation on appeals procedures in a composite document and ensure its ready accessibility through the OAQ website and other appropriate means. OAQ is working on an integrated document which covers all aspects of appeals. ESG 3.8 Accountability procedures (ENQA Criterion 7) Agencies should have in place procedures for their own accountability. Substantially compliant That OAQ develop and strengthen its procedures to ensure a continuous cycle for reviewing its own effectiveness at all levels. OAQ initiated a process with its supervising bodies (SUC and the Scientific Advisory Board) to implement these recommendations.

12 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 12 That OAQ develop a process for the on-going benchmarking of its own performance That OAQ review the effectiveness of the operation of its Scientific Advisory Boards That OAQ, as a matter of urgency, develop, in consultation with relevant entities, a governance structure That OAQ develop and publish a full strategic plan and institute a process of reviewing its performance against the targets in the plan The OAQ exploits the feedback of the Universities on the Quality Audits 2013/14, collected and discussed in the Q-Netzwerk, to develop further cycles of procedures. OAQ has developed and published a Strategy and Action Plan (see ESG 3.5). For each action, the internal version of the Action Plan includes a responsible, a deliverable, a milestone and an indication of the need for additional resources. The actions will be monitored internally and reported upon every year to the Scientific Advisory Board. Mechanisms for evaluating the effectiveness of the director as well as the Scientific Advisory Board were discussed with the SUC; technically these issues are outside the competence of the OAQ. However, the action plan to implement the strategy offers a frame of reference to evaluate the directors work. OAQ introduced time management software that allows the time we spend in our various tasks and functions to be evaluated and, if needed, to better distribute the tasks among the team. The activities of OAQ and the functions for each staff member have been formally described in the Organisation Handbook, including the new position of a deputy director Further actions to be determined The bylaws defining the new Accreditation Council and the future Swiss Agency for Accreditation and Quality Assurance under the new law HFKG will allow reviews of our own effectiveness at all levels (Director of the Agency, Advisory Board). OAQ plans to evaluate the effectiveness of the new methodologies introduced by the HFKG, by 2016

13 ENQA Review 2011: Empfehlungen und deren Umsetzung durch das OAQ / 13 ENQA Criterion 8 i. The agency pays careful attention to its declared principles at all times, and ensures both that its requirements and processes are managed professionally and that its judgments and decisions are reached in a consistent manner, even if the judgments are formed by different groups; ii. If the agency makes formal quality assurance decisions, or conclusions which have formal consequences, it should have an appeals procedure. The nature and form of the appeals procedure should be determined in the light of the constitution of the agency; iii. The agency is willing to contribute actively to the aims of ENQA. None None required