Sustainability in University Rankings. Rankings and Sustainability

Transcription

1 Sustainability in University Rankings 21 October 2013, Venice (Italy) Rankings and Sustainability Stefano PALEARI University of Bergamo, Rector and CRUI, President Results are from the research project: Le università nel nuovo secolo: nuovi modelli per una società in trasformazione Team Higher Education UnBG-CRUI

2 Our approach Sustainability issues are to be contextualized Why do we use measures (rankings) in Higher Education? Sustainability in Higher Education Measures (Rankings) and Sustainability 2

3 Why do we need measures for HE? Efficient allocation of limited resources Accountability towards funders and society Universities are (also) financed by the state Transparency of results Main World University Rankings: Academic Ranking of World Universities (ARWU) Quacquarelli Symonds (QS) Times Higher Education Ranking (THE)

4 ARWU Criteria Indicator Weight Quality of Education Quality of Faculty Alumni of an institution winning Nobel Prizes and Fields Medals Staff of an institution winning Nobel Prizes and Fields Medals Highly cited researchers in 21 broad subject categories 10% 20% 20% Research Output Papers published in Nature and Science* 20% Papers indexed in Science Citation Index-expanded and Social Science Citation Index 20% Per Capita Performance Per capita academic performance of an institution 10% Total 100% 4

5 QS Criteria Indicator Weight Teaching Faculty Student Ratio 20% Reputation Academic Peer Review from global survey 40% Recruiter Review from global survey 10% Research Citations per Faculty from Sciverse Scopus 20% International Orientation Percentage of intanational students 5% Percentage of intanational staff 5% Total 100% 5

6 THE Criteria Indicator Weight 2011 Weight 2012 Reputational survey - teaching 15% 15% = PhD awards per academic 6% 6% = Teaching - the learning environment Undergraduates admitted per academic 4,5% 4,5% = Income per academic 2,25% 2,25% = PhD awards/ bachelor's awards 2,25% 2,25% = Citations Research Citations impact (normalised avarage citations per paper) from Scopus 32,5% 30% Research - volume, income and reputation Reputational survey - research 19,5% 18% Research income (scaled) 5,25% 6% Papers per academic and research staff 4,5% 6% International Outlook - staff, students and research Industry Income - innovation Ratio of international to domestic staff 0,75% 2,5% Ratio of international to domestic students 3% 2,5% Proportion of internationally co-authored research papers Research income from industry (per academic staff) 2% 2,5% 2,5% 2,5% = Total 100% 100% 6

7 Worldwide University Ranking Methodological issues Subjective measures i.e.: Subjectivity of weights and indicators Biased measures Rankings favour hard science and publications in English language Reputation vs. Contemporaneous performance measures Advantage for big universities Advantage for historical universities (Nobel prize winners) Measures affected by exogenous factors Total income of institutions 7

8 World University Ranking Correlation Spearman coefficient: d i : difference among the position of the i university n: number of universities listed High level of correlation among top-100 ranked institutions (Sample: 137 universities listed in all the three rankings) ARWU 2011 THE 2011 QS 2011 ARWU ,000 THE ,684 1,000 QS ,421 0,614 1, Correlation ARWU 0,994 THE 0,957 QS 0,989 Spearman correlation Matrix in year 2011 and among different rankings and among the same ranking in 2011 and

9 Best University according to THE 1/2 Top 10 universities in Times Higher Education Ranking 2012 Rank Università Students Faculty Student / Faculty 1 California Institute of Technology University of Oxford Stanford University Harvard University Massachusetts Institute of Technology (MIT) Princeton University University of Cambridge Imperial College London University of California, Berkeley University of Chicago Top 10 universities in Times Higher Education Ranking 2012 Average students number "top 10" Average students per faculty "top 10" 7 Avarage students per faculty in Italy 30 9

10 Best University according to THE 2/2 Top university by country in THE 2012 Istituzione Studenti Docenti Student to faculty ratio % studenti internazionali % graduate students California Institute of Technology ,0 25,0% 55,4% University of Oxford ,8 29,2% 35,8% ETH Zürich ,3 33,1% 50,2% University of Toronto ,7 10,3% 18,8% University of Tokyo ,9 7,6% 49,5% University of Melbourne ,5 28,5% 27,3% National University of Singapore ,8 36,3% 21,0% University of Hong Kong ,2 30,0% 41,7% Karolinska Institute ,1 7,1% 51,6% Peking University ,2 5,6% 51,0% Ludwig-Maximilians-Universität München ,8 14,1% 14,0% Pohang University of Science and Technology ,5 1,6% 48,0% Katholieke Universiteit Leuven ,8 15,5% 47,2% École Normale Supérieure ,4 11,1% 77,8% Utrecht University ,4 6,3% 12,9% ,5 17,4% 40,1% 10

11 Europe vs. US Leading position of US universities: 17 out of 20, and 57 of the top 100 in the ARWU 2011/2012 Considering the top 500 universities, Europe with 202 overtakes America with 182 top-ranked institutions In Europe, with a predominant focus on broad-based and regionally distributed support for a fairly large number of universities, resources were more evenly distributed Key issue: What is Excellence? E.g. the University of Göttingen does rank very highly in rankings, but it is not successful in the German Initiative on Excellence 11

12 Why do current rankings fail in measuring sustainability? Use of a sole indicator to evaluate multi-output organisation They do not consider emerging aspects as sustainability and diversity of institutions They measure only measurable facts Measure only what is measurable, not all what matters Describe the whole only through a part In sports F1: who determines the result? the driver, the car, the team Does one component really represent the whole? 12

13 Do rankings improve (a fair) university competition? If universities are to challenge, rankings should refer to universities in the same league, By contrast they do not discriminate: Different budget Different recruitment procedures Human capital is the most important productive factor into a university! Different tuition fees Once again in sports NBA: the same salary cap for all the teams (with fines for those not complying) 13

14 Funding of European HE Many countries reduced public funding to their tertiary education systems from the beginning of the crisis Reduction of funding mainly in the countries in the south and east of Europe Trends in Public Funding to Higher Education Europe over the period Source EUA s Public Funding Observatory (June 2012) 14

15 Funding per citizen ( ) Country * Including EU Funds Funding (mln ) Population 2011 ('000) Funding per citizen Change Fonte Elaborazione propria dati EUA s Public Funding Observatory e Word Bank Statistics Change Inflation-adjusted Norway ,0% 21,0% Sweden ,0% 21,0% Germany ,0% 20,0% France ,8% 6,4% Iceland ,0% 7,2% Ireland ,0% -21,0% Austria ,0% 13,0% Netherland ,0% 7,5% Spain* ,5% -11,0% UK ,0% -13,0% Italy ,0% -14,0% Croatia ,3% 1,8% Slovakia ,1% -1,5% Poland* ,0% 8,6% Czech Reupublic ,0% -17,0% Lithuania ,0% -22,0% Portugal ,5% -4,1% Hungury ,0% -24,0% Greece ,0% -25,0% Belgium - French Community 585 n.a. 0 19,0% 16,0% 15

16 Funding development: Italy Fonte EUA s Public Funding Observatory 16

17 Funding development: other countries Fonte EUA s Public Funding Observatory 17

18 Do rankings help universities to work more efficiently? If rankings have to increase efficiency: They need to be an instrument, they should not be the objective! Universities should be empowered the instruments to correct their performances i.e.: more autonomy in recruitment 18

19 Italy: Consequence of funding drop To do more with less? Sharp declining of academic staff Permanent academic staff went back to the level of 2001 Dramatic declining of non-academic staff Increase in Student to Faculty Ratio State universities Δ Δ % Bachelor & Master students n.a ,3% Tenure Academic staff ,9% Full Professors ,0% Associate Professors ,0% Tenure Researchers ,3% Non-tenured researchers ,2% Non-academic staff ,2% Studenti to Faculty Ratio n.a Source: Reprocessing data from MIUR e CINECA database 19

20 Something new: the U-Multiranking Advantages Better transparency of performance evaluation Measure excellence in different missions and levels User-oriented Evaluation not only at institutional level, but also at cource level Drawbacks: Still developing (first release in February 2014) Costly for institution to collect and provide information and data 20

21 How can we improve rankings? Efficiency oriented Including at least one measure which considers both inputs (staff, funding) and outputs Importance of network What does really matter? To be the first, or to add value to the system? Importance of diversity Promote best practices, rather then the use of synthetic (and biased) rankings 21

22 How can we improve the HE system as a whole? Set the «ultimate goal» for the HE system, and set rankings accordingly Is the HE system designed to train the top students, or to improve the quality of the system as a whole? Is the HE system designed to mitigate the tendency to inequality in the social system? 22

23 Conclusions Sustainability is Efficiency The role of the university in the society Competing in the same field (classifying universities) Competing with the same rules Setting long-term goals 23