Times Higher Education World University Rankings: what quality measures?

Transcription

1 Times Higher Education World University Rankings: what quality measures?

2 Cover: Background to the World University Rankings History Impact Some student perspectives WUR Methodology Indicators and measures

3 Background to the World University Rankings

4 Times Higher Education First issue of supplement World University Rankings launched THE becomes standalone publication independent of The Times THE relaunched in magazine format World University Rankings production moved to Thomson Reuters World University Rankings brought in-house

5 A brief history of the World University Rankings st edition of World University Rankings commissioned by Times Higher Education 2015 Times Higher Education move production in-house Switch to Elsevier for bibliometrics 2010 Improved methodology Move to Thompson Reuters Universities deserve a rigorous, robust and transparent set of rankings

6 Impact of Rankings Rankings prompt change in areas that directly improve student learning US Institute for Higher Education Policy (IHEP). May 2009 Rankings "help by encouraging more informed policy making... they can stimulate national debate and focused analysis... which in turn may lead to positive policy changes at system level. European Universities Association, April 2013 The data collected for the THEWUR provide a useful set of indicators which enable us to analyse the dynamics of higher education development and to comparatively relate excellence to policies Dirk Van Damme, Head of the Innovation and Measuring Progress Division (IMEP) at the OECD The THE rankings are the principal yardstick we should look to Shashi Tharoor, former Minister of State for Human Resource Development, India. Times Higher Education rankings now increasingly seen as the gold standard. Ferdinand Von Prondzynski, Vice Chancellor, Robert Gordon University

7 Impact of Rankings More universities [compared to earlier study] monitor their peers worldwide (77% vs nearly 50%) and that an "overwhelming majority" use rankings to inform strategic decisions Hazelkorn Rankings and the Reshaping of Higher Education Hazelkorn 2011: the evidence is compelling that universities do use rankings as a strategic tool and a management instrument Rankings have become an explicit part of target agreements or contracts between presidency and departments and featured in individual faculty performance contracts. Rankings have taken the function of data collection and analysis out of the back-office, and placed it at the centre of strategic decision making and performance measurement.

8 Rankings and Key Performance Indicators We have not targeted an actual ranking position in our strategic plan, but we have 20 key performance indicators and there is a connection with rankings -- several match the THE rankings indicators Anton Muscatelli, principal, University of Glasgow UK A majority of institutions stated that they had responded to league tables in the areas of KPIs and, in many cases, used the league tables variables to review these indicators. Case study pre-1992 university some of the key performance indicators used by Council [ the Universities governing body] now are modelled to dovetail with elements of the rankings. Counting what is measured or measuring what counts? League tables and their impact on Higher Education in England - HEFCE

9 Some Student Perspectives

10 Student perspectives decision factors Teaching quality How important are these factors when deciding which university to apply to? Answers 6 and 7 on a 7 point scale; 7= Strongly Agree Subject available Tuition fees Academic reputation rankings Living expenses Safety of campus / local area Scholarships Entry requirements Responsiveness of university in Source: Hobsons Beyond the data: Influencing international student decision making May 2014 Links with employers (internships) Graduate employment rates The ever-increasing usage of rankings [by students] is fast becoming a recurring theme. Hazelkorn

11 Student perspectives teaching quality factors Academic reputation / ranking When thinking about teaching quality, how important are the following factors? Answers 6 and 7 on a 7 point scale; 7= Strongly Agree Subject / course ranking Student satisfaction with university Tuition fees Use of technology in teaching Academic profile of lecturers Student satisfaction ranking Source: Hobsons Beyond the data: Influencing international student decision making May 2014 Links with employers Quality of facilities % student in employment / further study after graduation Teaching hours per week

12 Student perspectives best teaching quality? Conjoint factors of importance Tuition Fee Shown 8-10 sets of institutions with random combination of levels for each attribute and asked to choose which one represented best teaching quality Subject ranking University ranking % students in employment Source: Hobsons Beyond the data: Influencing international student decision making May 2014 Teaching hours per week Student satisfaction

13 Student perspectives The quality and impact of our research is officially acknowledged. In the 2014 Research Excellence Framework, 62 per cent of our research submissions were classified as worldleading or internationally excellent From a UK university international student application webpage

14 World University Rankings: Methodology and Data Sources

15 THE WUR methodology Source 13 metrics 5 areas/pillars Ranking Width of line corresponds to weighting of metric

16 Data sources: portal 1667 institutions invited 1126 participated Details at institutional and subject level

17 Data sources: reputation survey Survey of academics selected by Elsevier Balanced by geography OECD data 10,000 responses

18 Data sources: Scopus Move from Thompson Reuters Web of Science to Elsevier s Scopus Three measures Field Weighted Citations to avoid subject bias

19 THE WUR Universe 1,667 1, Institutions approached Institutions that submitted data Institutions with sufficient publications Institutions that have undergraduates Institutions with sufficient data Ranked More Universities than ever before 88 countries involved 70 countries with ranked institutions 801 Universities will be ranked with banding

20 The Indicators: quality measures?

21 3 Ps model of teaching and learning J B Biggs, 1993 Higher Education Research and Development Presage Exists before a student starts learning or being taught Process What is going on in teaching and learning Product Outcomes of the educational process

22 Dimensions of quality Graham Gibbs, HEA 2010 Presage Funding Student Staff ratio Quality of teaching staff Quality of students Process Class size Contact hours Quality of teaching Research Environment Intellectual challenge Assessment & Feedback Other - reputation Product Student Retention Performance and degree classification Employability

23 THE WUR methodology

24 Teaching Presage

25 Teaching Presage Process

26 Research Process

27 WUR - less able to capture Quality of teaching staff, quality of students [Presage] Class sizes and contact hours, assessment and feedback, quality of teaching (qualifications / students judgment), level of intellectual challenge (curriculum, depth, engagement) [Process] Student retention, performance/degree classification, employability [Product]

28 Challenges of choosing indicators and sourcing data Relevant and valid - globally Clear sense of what is better performance Transparent and easily understood Available - globally Independent source or straightforward for institutions to report

29 Conclusion WUR aims to capture broad performance of universities and not just teaching quality Rankings are one part of the approaches to the assessment of higher education and bring into focus the indicators included Used by students, institutions, funders, government bodies THE is open to discussion and suggestions about how the element of teaching quality in World University Rankings could be enhanced

30 Thank You