U Multirank Ideas and Realisation of a multi dimensional, user driven Ranking Professor Dr. Frank Ziegele Rankinger Konferanse, Bergen, November 2013 /U Multirank /U Multirank 1
3 issues will be covered Why and in which way is U Multirank different? What is the benefit of U Multirank and how does it work? What is the state of play? 2
Why do we need U Multirank? There is a market for commercial rankings, but they are not able to sufficiently serve relevant needs focus on research and reputation: only relevant for one type of university not enough information about different profiles and for different needs of stakeholder groups (esp. students) problematic effects of league tables on HE policies easily measurable data, misleading composite scores U Multirank will avoid all this and is meant to be an instrument which really helps to make decisions, which allows an analysis of strengths and weaknesses, which takes into account the diversity of institutional missions and individual preferences 3
What are the distinctive features of U Multirank? Multidimensional ranking Going beyond the traditional focus on research excellence Five dimensions: teaching & learning, research, knowledge transfer, international orientation, regional engagement (up to around 40 indicators) No composite indicators, no pre defined weights on indicators, single ranking for each indicator User driven ranking Personalised ranking allows users to rank by their own preferences and priorities on dimensions and indicators ( democratised approach) flexible web tool 4
What are the distinctive features of U Multirank? Comparing like with like Link to mapping indicators allowing identification of institutions with similar institutional profiles Multi level ranking Combining institutional ranking (whole institutions) and field based rankings (starting now with: electrical and mechanical engineering, business studies, physics) Stakeholder oriented processes Intensive inclusion of stakeholders in development and continuous refinement of U Multirank Independence creation of a governance and funding model ideally ensuring independence from political agendas, interests of actors and market forces 5
With this approach U Multirank will create multi dimensional performance profiles Teaching and Learning Institutional ranking Field based ranking Student staff ratio Graduation rate (BA and separately MA) Percentage of academic staff with PhD Percentage of students graduating within normative period (BA and separately MA) Rate of graduate employment Inclusion of work experience in degree programme 6
Teaching and Learning Student Satisfaction Indicators Institutional ranking Field based ranking Overall learning experience Quality of courses & teaching Organisation of the programme Contact to teachers With this approach U Multirank will create multi dimensional performance profiles Social climate Facilities (libraries, laboratories, rooms, IT) Research orientation of teaching /programme Inclusion of work experience /practical elements 7
With this approach U Multirank will create multi dimensional performance profiles Research Institutional ranking Field based ranking External research income (per fte academic staff) Doctorate productivity Total research publication output (per fte academic staff)* Art related output Field normalised citation rate* Highly cited research publications * Interdisciplinary research publications* Research orientation of teaching (student survey) Number of post doc positions 8
With this approach U Multirank will create multi dimensional performance profiles Knowledge Transfer Institutional ranking Field based ranking Income from private sources (service contracts, consultancies, licenses, royalties, trials, etc.) Joint research publications with industry* Patents (per fte academic staff) Co patents with industry (per fte academic staff) Number of spin offs (average over three year period) Patent citations to research publications* Revenues from Continuous Professional Development 9
With this approach U Multirank will create multi dimensional performance profiles International Orientation Institutional ranking Field based ranking Educational programmes (BA/MA) in foreign language International orientation of degree programmes Opportunities to study abroad (student survey) Student mobility (composite of incoming, outgoing, joint degree students) Percentage of international academic staff Percentage of PhDs awarded to foreign students International joint research publications* International research grants 10
With this approach U Multirank will create multi dimensional performance profiles Regional Engagement Institutional ranking Field based ranking Percentage of graduates working in the region Student internships in regional enterprises Degree theses in cooperation with regional industry Regional joint research publications* Income from regional sources 11
3 issues will be covered Why and in which way is U Multirank different? What is the benefit of U Multirank and how does it work? What is the state of play? 12
U Multirank will provide specific benefits to participating institutions Basic benefits Visibility of institutions with different profiles, marketing use Visibility of specific profiles: tool to look for similar institutions Comparison of performance with like Allowing differentiated SW analysis, evidence based discussions both on institutional and on field level Additional analysis of own data Compared with the total sample Institutional & field based data Detailed analysis of student survey Support for benchmarking processes Either for networks or organisations Multi level: institutional and fields 13
Web tool prototype The best way to get an impression of the benefits for the users is to see how it works! (Prototype) 14
3 issues will be covered Why and in which way is U Multirank different? What is the benefit of U Multirank and how does it work? What is the state of play? 15
Major steps have been made in 2013 Final quality check and refinement of indicators, development of data collection tools and quality assurance Communication within the HE community Stakeholder consultations (indicators, web tool) Development of web tool Recruitment of participants goal 500, almost 700 inscribed, some will still drop out almost 80 countries, all continents good mix of all kinds of profiles currently 11 from Norway 16
Data collection and more activities take place now Data collection (surveys) incl. intensive communication with participants Bibliometric analysis (incl. all top research intensive universities) Data quality control and data analysis processes Transformation of web tool prototype into functioning and designed tool 17
Data collection and more activities take place now Development and implementation of a communication strategy to reach the potential users Feasibility study on integration of non university research organisations Proposals on long term governance structures and funding models for U Multirank First release March 2014 18
U Multirank Ideas and Realisation of a multi dimensional, user driven Ranking Thank you for your attention! Professor Dr. Frank Ziegele frank.ziegele@che.de /U Multirank /U Multirank 19
The web tool offers a variety of access points 20
Students choose a field and then have the chance to make some simple choices on the desired university profile (reducing the matching institutions) 21
Here is such a reduced number of institutions matching the desired profile. 22
For the matching institutions a first table with teaching oriented indicators is presented. This is just a starting point, students could vary in numerous ways. 23
For each indicator there is an explanation, telling the student what it says. 24
The table could be sorted by the performance for a specific indicator. Or the dimensions could be extended. 25
Change Indicators leads to the whole list of indicators available for the field, the preferred indicators could be clicked and displayed in a new table. 26
Or the institutions could be filtered by continent/country. 27
The second option to enter the system is your ranking, meaning you as the user build your own ranking (similarities to the student track, but more expert oriented). 28
The like with like comparison requires the definition of a profile first. Mapping is now more complex. 29
For the defined profile you get an overview over all dimensions with 3 indicators per dimension. But again you could expand to the full set. 30
And again you could change the indicators in the table. 31
Not only can you compare like with like, but you can also compare a particular institution with others. 32
You select one specific university (probably your own ) to be compared with similar ones. 33
Then the mapping is pre filled with the profile of the chosen university. You could ignore items. 34
And you will again get a ranking table with 3 indicators per dimension, with the chosen institution now on top. 35
There are not only user driven rankings, but we provide ranking lists of specific subsets of institutional profiles, presented as our rankings. These lists are static. 36
One of those pre defined rankings will be the ranking of research intense universities. The indicators shown are adapted to the profile (here: research orientation). 37
If you want to take a detailed look at a university you could choose Explore Profiles. 38
A university has to be selected. 39
The detailed profile is shown, including all ranking indicators as well as information about the institution. 40