Rankings and competitive metrics from a university perspective

Transcription

1 Rankings and competitive metrics from a university perspective May 06, 2014 Björn Forslöw, Director s Office Details are hidden on a few slides, two slides with bibliometric maps are excluded, due to licence agreements. Certain data included herein are derived from the Web of Science prepared by THOMSON REUTERS, Inc. (Thomson ), Philadelphia, Pennsylvania, USA: Copyright THOMSON REUTERS 2014 All rights reserved. The network was visualized with Gephi, an open-source graph engine

2 Today s presentation - content Background Do rankings tell us something (significant)? Examples of tailor made tools for monitoring Collaboration and publication mapping Bibliometric performance comparisons 2 13 maj

3 Organisation of intelligence monitoring and analysis at KI (synoptic) Management Team Board of PhD Edu. Board of Research Board of Education KI/SLL Clinical development Projects University Library Databases Bibliometrics Director s Office Rankings Benchmarking Method. support Risk Managem. Communications Office Media monitoring Media analyses Faculty Office and international Relations International Board Support Research Support office Grants Funding Innovations 13 maj

4 Are metrics and science indicators indispensable? What are the drivers? Globalization and increased competition Transparency on quality and output requested by stakeholders Planning and organizational development Incentives and resource allocation Communication and marketing Increased data availability more applications requested just because it s possible 13 maj

5 Rankings often play a paradoxical role how are they percepted in different target groups? Target groups Students the most frequent readers Deliveries In general - poor education indicators Scientists don t use rankings Research indicators dominate rankings Opinion makers use simple numbers Multi-indicators, field tables Publicerad : 2 oktober 2013 Allra högst rankat av de skandinaviska universiteten är Karolinska Institutet, som dessutom klättrar från plats 42 till 36 i årets rankning.. Men de allra viktigaste jämförelserna för oss är de som analyserar medicinska universitet, universitet med samma uppdrag som vi har säger Anders Hamsten. 13 maj

6 KI is frequently evaluated, a selected monitoring is required International Domestic High priority 5 Medium priority ~16 High priority 8 Medium priority ~15 In-house High priority 11 Medium priority ~7 Total: >60 different evaluation frames 13 maj

7 Karolinska Institutet international ranking positions 2013/2014 Academic Ranking of World Universities, ARWU 2013 Total rank world Total rank Europe Clinical Medicine & Pharmacy world rank Clinical Medicine & Pharmacy Europe rank Agriculture and Life Sciences world rank Agriculture and Life Sciences Europe rank KI position Times Higher Clinical, Clinical, Education World Preclinical & Preclinical & University Rankings 2013* Total rank world Total rank Europe Health world rank Health Europe rank KI position Total rank world (only Total rank Europe (only Pharmacy & Pharmacy & QS World University multi-faculty multi-faculty Medicine Medicine Pharmacology Pharmacology Rankings 2014 universities) universities) world rank Europe rank world rank Europe rank KI position N.A. N.A * = Officially named 2013/2014 Whole Universities Subjects or Fields 13 maj

8 Different methodologies different results!? Indicators and Weights for ARWU Criteria Indicator Code Weight Quality of Education Alumni of an institution winning Nobel Prizes and Fields Medals Alumni 10% Quality of Faculty Staff of an institution winning Nobel Prizes and Fields Medals Highly cited researchers in 21 broad subject categories Award 20% HiCi 20% Papers published in Nature and Science* N&S 20% Research Output Papers indexed in Science Citation Index-expanded and Social Science Citation Index PUB 20% QS subject tables - weights Per Capita Performance Per capita academic performance of an institution PCP 10% Total 100% * For institutions specialized in humanities and social sciences such as London School of Economics, N&S is not considered, and the weight of N&S is relocated to other indicators. Bibliometrics Reputation surveys Efficiency Ratios Awards 13 maj

9 Small numbers big changes! Ranking positions are vulnerable Total index score of European top15 Universities ARWU 80 73,6 Total Index Score ,7 43,7 42,6 41,7? 32,9 32, ,8 31,4 31,3 31, , Univ Cambridge Univ Oxford Imperial Coll London Univ Coll London Swiss Fed Inst Tech - Zurich Univ Utrecht Karolinska Inst Stockholm Univ Paris 06 Univ Edinburgh Univ Munich Tech Univ Munich Univ Manchester Source: Institute of Higher Education, Shanghai Jiao Tong University, Academic Ranking of World Universities (ARWU), 2005 Univ Copenhagen Univ Zurich Uppsala Univ 13 maj

10 Linear ranking order or die tossing within tiers of equals? THE total ranking all categories 13 maj

11 Rankings based entirely on reputation are of little value Regarding the impact of reputation-based rankings, as the reputation-based score dwindles rapidly from the first top-ranked university down to the 50th, the reputation indicators have a very limited impact on the THE World University Ranking. This implies that rankings based entirely on reputation are of little value. The very steep reputation-based curve means that the influence of reputation is substantial for the first few most highly ranked institutions but quickly decreases in significance thereafter. Source: Andrejs Rauhvargers: Global university rankings and their impact, EUA maj

12 The THE Reputation Ranking index distribution is bisectional 13 maj

13 Can we use ranking tables as proof of long term progress? 13 maj

14 Can uncompounded indicators tell us more? 13 maj

15 Each institution needs to find its own set of tools and indicators to monitor and analyse Databases International Research and Education agreements International research collaboration mapping Research publication output benchmarking 13 maj

16 13 maj

17 The Karolinska bibliometric infrastructure Web of Science Verification Karolinska Institutet + SLL University Hospitals SQL database Statistics, Mappings, Evaluations Medline MeSH-terms 13 maj

18 Search KI articles Overview and Refine Link to Web of Science Verifying KI authors in green 13 maj

19 Karolinska Collaboration Map 13 maj

20 Karolinska Collaboration Map xxxxx xxxxx 13 maj

21 Agreement list Singapore 13 maj

22 Copublications KI-A*STAR Limit: Authors with more than 3 copublications, MeSH headings stronger than 4 13 maj 2014

23 Copublications in Medicine and Life Science Nordic universities Publication years Low filter: 50 copublications 13 maj 2014

24 The broad field of investigation can be defined by MedLine and accessed by Web of Science WoS/Medline overlap Providing data to this study covers 4800 journals and magazines Web of Science 1995 and onwards Provider of basic data to KI publication database covers journals and Magazines (2012) Medline 1995 and onwards covers 5200 journals and Magazines (2012) November

25 Important definitions and delimitations Investigated institutions Universities are defined as medical centers including associated hospitals Document types Only articles and reviews are considered (peer-reviewed) Counting methods Full count (not fractionalized by the number of co-authors) 13 maj

26 A selection of bibliometric indicators P, No of Publications Cf, field normalized citation score ( average citation impact ) Sum of Cf, Cf of each paper aggregated ( total, size and quality dependent impact ) The normalization of the citation indicator is used to compensate for differences in citation pattern between research areas, document types and publication years Molecular biology, Surgery Original Article, review, letter, note 1995, 2011, 13 maj

27 A bibliometric trend for selected International and Nordic universities Selected International and Nordic institutions Medicine/LifeScience (MedLine delimited) Y-axis: Average Cf X-axis: Publications Size: SumCf Colour: Share Top5% 13 maj

28 Quality adjusted publication output ( field normalized total citation impact ) Selected International and Nordic institutions Medicine/LifeScience (MedLine delimited) Univ A Univ B Univ C Univ D Univ E 13 maj

29 Thank You for your Attention!