Promoting the use of graduate surveys. Catalan University Quality Assurance Agency tale.

Transcription

1 Promoting the use of graduate surveys. Catalan University Quality Assurance Agency tale. Dr. Anna Prades Ljubljana, Slovenia, 27th September

2 Introduction: my context (1/4) The purpose of AQU Catalunya is the assessment, accreditation and certification of quality in the sphere of the universities and higher education centers of Catalunya. Graduate s survey aim Embedding graduate survey indicators into internal quality assurance systems. 2

3 Introduction: my context (2/4) 12 UNIVERSITIES: 7 public + 5 private Vic (1) Girona (1) Students +/- 225,000 Graduates +/- 22,000 Population % students in public universities 7 million +/- 85% Lleida (1) Barcelona (8) Tarragona (1) Supply Catalunya 11.0%

4 Introduction (3/4): the Catalan Graduate survey Why? How? Who collects the data? Who analyses it? Who participates? What do they receive? To benchmark indicators in quality assessment processes (since 2001) Designed by a committee with experts from each Catalan university (in 2000). Universities provide registers. AQU (contracts a telemarketing company; CATI system) AQU provides an overall report, as well as descriptive targeted reports (via an SQL application) All Catalan public and private universities Everything is public: overall results, targeted reports Databases: universities (their own). Full DB to academic community.

5 Introduction (4/4): Technical features Population Sample Response rate Sample error SURVEY Promotion 98 SURVEY 2005 Promotion , , SURVEY 2008 Promotion , SURVEY 2011 Promotion , Descriptive results online: Databases available to the academic community

6 Keywords to frame this discussion Accountability Transparency Reporting Bureaucritasion Using data in decision making Data-informed leadership Continuous improvement PDCA cycle Professionalization of work related to quality Self-regulation Uncertainity Change Marketing Quality awareness Learning organizations Quality culture Rankings Organizational change

7 Promoting the use of information in 5 steps 3 Embedding indicators in IQAS and EQAS Coaching the coachers 2 Targeted Reporting Collecting data / Rigorous analysis 1 Designing: to whom, what for 7

8 STEP1 : Designing. To whom and what for (1/3) Supply and demand adjustment Taking decisions Career services Program profile design. Syllabus Practical dimension design Future students Administration / University government STAKEHOLDERS Teaching staff Students Employers Career advisers/ Tutors University government

9 STEP1 : Designing. To whom and what for (2/3) Improve Supply & Demand Program specification design Career services Plan / Design Check /monitor Check Do / Develop Planning stage Employment rate, education-job match, stability rate, salary Competencies acquired vs. required, Job functions, branch of economic activity Work during studies, hiring factors, competencies required Supply and demand adjustment Do we need so many graduates? Are they necessary to our society? Do we need this degree? Are its graduates necessary to our society? Program specification design What kind of graduates do we need? Should we design a generalist degree or one highly specialized? Which kind of internships do we need? At which kind of enterprises? Which competencies will they need? Which functions they will need to develop? Career services What could be helpful to students to know in order to gain a qualified employment? Which kind of enterprises employ our graduates? How to gain access to the labour market? Which were the main factors for being contracted among the previous graduates?

10 STEP2 : Transparency is too much information?

11 STEP2 : Targeted reporting 20 pwt 4 p. 113 p. Dèficit de formació pels graduats que realitzen funcions de nivell universitari en el seu lloc de treball Formació teòrica Formació pràctica Lideratge Treball en equip Comunicació Oral Comunicació escrita Creativitat Pensament critic Presa de decisions Solució problemes Informàtica Idiomes Documentació Gestió 4 p. -3,20-2,80-2,40-2,00-1,60-1,20-0,80-0,40 0,00 0,40 0,80 1,20 1,60 2,00 El dèficit s'ha calculat restant la mitjana de cada competència que es necessària per desenvolupar la feina, de la mitjana de l'avaluació del nivell de formació rebut a la universitat. Només inclou el colectiu de graduats que desenvolupen funcions de nivell universitari.l 4 p 11

12 STEP 3 : Embedding indicators in EQAs Assessment process Ex ante accreditation Issue Criterion 2: Justification (out of 10) states. Use of employment indicators is encouraged. Where relevant, the instituion must account for the vacational applicability of the study. Monitoring Institutions have to use evidence (such as graduate labour market outcomes) to evaluate the development of the degrees program. The degree of evidence- driven enhancement measures is assessed. Accreditacion (expost) In draft Employability (within sustainability criteria). It will be assessed using three indicators from the graduate survey: employment tax (vs LFS for the same age span), quality of employment, and average appraisal of their undergraduate studies. 12

13 Thank you! 13