THE RANKING WEB NEW INDICATORS FOR NEW NEEDS. 2 nd International Workshop on University Web Rankings CCHS-CSIC, Madrid (Spain).

Transcription

1 2 nd International Workshop on University Web Rankings CCHS-CSIC, Madrid (Spain). April 21 st 2009 THE RANKING WEB NEW INDICATORS FOR NEW NEEDS Isidro F. Aguillo Cybermetrics Lab. CCHS-CSIC

2 AGENDA Why Ranking? Why Web? Comparison with other Rankings Ranking Web Future Developments The Floor is Open 2

3 ABOUT US Spanish Research Council Largest public research body in Spain 130 institutes (mostly basic science) 20% of the Spanish scientific output 2 Nobel Prizes Cybermetrics Lab Research group at the Center of Humanities and Social Sciences Editors of ejournal Cybermetrics EU Projects involved Staff: 4 researchers, 1 technician 3

4 TO RANK OR NOT TO RANK Global market Explosive international student mobility Growing demand of information Prior to 2003 there were no World Universities Rankings Demise of national rankings Impact of Shanghai s ARWU Trend towards increasing complexity Not obvious results, multiple classifications, confusing figures Student oriented Web publication reached World audiences League tables are easy to understand Unexpected results, rectors & politicians unaware of the actual situation But Methodological problems Mostly research based Promoting ranking-improvement tactics 4

5 STUDENT MOBILITY Countries of origin Countries of destination USA UK Australia Japan Germany Canada France Total Total from Asia of which, from Eastern Asia of which, from South-central Asia of which, from South-eastern Asia of which, from Western Asia Total from all countries, Total from all countries, Growth in internationalization of tertiary education ( ) Source: Education at a Glance 2008: OECD 5

6 HOW TO RANK? Indicators a 1. faculty/student a 2. funding a 3. papers published a 4. citations received a 5. degrees offered a 6. prizes a 7. external reviews a 8. web size a 9. intl. faculty a 10. foreign students a 999. computers ratio WeightsRank=w1*a1+w2*a2+ w w w i 1 w i =1 A methodologically sound approach Considering many indicators, not only outputs but also inputs based Applied a multivariate non parametric analysis to estimate w i 6

7 FEASIBILITY PROBLEMS No data available for most of the countries Especially in developing ones in Africa, Asia and Latin America Data not comparable due to the lack of standards Professor/researcher/student/international student Poor ability to discriminate Ratios with low numbers Nobel Prizes (few universities with 2 or more) Bibliometric biases Biomedicine Social & Humanities, Technology North America, Europe Rest of the World English Other languages 7

8 WHY THE WEB? The Web is already the main tool for scholarly communication The Web reaches and it is accessed from every country of the World The Web could reflect all the missions of the universities Research Teaching Third mission The Web provides easy access to relevant indicators Web visibility has proved its ranking capabilities Google PageRank 8

9 THE RANKING WEB Original aim Promote Open Access Initiatives Target: Web Publish or Perish Political agenda Reaching developing countries Including all the missions, not only research Methodology Search engines as intermediaries: Easy way to collect data Link (quantitative) analysis as a proxy of quality assessment Impact Factor based composite indicator Consequences Webometrics Rank as an indicator: High correlation with other Rankings Academic Digital Divide Web (naming) bad practices: Taking the Web seriously 9

10 AN ACADEMIC MODEL Activity (50%) Impact (50%) Size Rich files Scholar Visibility Number of Webpages 20% Number of documents 15% Number of papers 15% Number of external inlinks Virtual referendum 10

11 WEBOMETRICS RANKINGS Published since 2004, inspired by the Shanghai Ranking Top 6,000 universities from a Directory with 17,000 entries Top 2,000 research centers from a Directory with 7,000 entries Two editions (January & July) per year Only organizations worldwide with their own web domain or subdomain Based on web indicators (activity and impact) 11

12 12

13 SUMMARY REGION / countries Top100 Top200 Top500 Top1000 Total NORTH AMERICA USA EUROPE United Kingdom Germany Italy France Spain ASIA Japan China & HK Taiwan Korea LATIN AMERICA Brazil Mexico OCEANIA Australia ARAB WORLD AFRICA WORLD

14 EXPECTED RESULTS North America Europe MIT (1 st region & world), Stanford (2 nd ) and Harvard (3 rd ) Universities University of Toronto (24 th world) Cambridge (1 st region), ETH Zurich (2 nd ) and Oxford (3 rd ) Universities Asia Tokyo (52 nd ), Taiwan National (55 th ) and Peking (117 th ) Universities Oceania National University of Australia (48 th ) Latin America Universidad Nacional Autónoma de México (44 th ) Universidade de Sao Paulo (87 th ) Africa University of Cape City (359 th ) 14 WR (Enero 2009)

15 MORE RESULTS Bad naming practices Universities of Catalonia (.es,.edu.,.net,.cat) Old and new domains (Imperial, Illinois, Victoria, Northwestern, Case Western, Pontificia de Chile, ) Common domain Splitted domain Japanese, French, Italian Jussieu campus (Paris 6, Paris 7, CNRS) Strasbourg Universities (3 under common domain) University of Helsinki (shared with the City Hall) Johns Hopkins University Universidad de la República (Uruguay) University of Zagreb Warsaw University Websites mostly in local language (Few pages in English) Biases Against Biomedical small size organizations Favoring Technology oriented and Distance Learning 15 based universities

16 COMPARING WORLD RANKINGS CRITERIA WEBOMETRICS ARWU HEEACT QS-THES Spain (research lab) China (university) Taiwan (gov agency) United Kingdom (firm) Univs analysed Univs ranked Teaching Alumni Nobel 10% Students/Staff 20% Internationalisation % Foreign Students 5% % Foreign Professors 5% Size Webpages 20% University 10% Research Documents 15% Nature&Science 20% Papers S&SSCI Papers GS 15% Papers S&SSCI 20% 20% Impact Links 50% Highly Cited Researchers 20% Prestige Faculty Nobel 20% Highly Cited Res Cit High IF Journals 50% Citations S&SSCI 30% Citations Scopus 20% Academic Survey 40% Employers Survey 10% 16

17 ADVANTAGES & DISADVANTAGES ARWU QS-THES HEEACT WEBOMETRICS Focus on excellence Perceived prestige Recent performance Developing countries Historical achievements Long term performance Other missions than research Estability Internationaliza tion Only research Normalization problems Easy to build Only research Bibliometric biases Discriminatory capability Prizes assignation University Hospitals Representation bias of the survey Size related Not stable Ties (beyond rank 200) Private company Size related problems No crown indicator Sensible to URL bad practices Google Scholar data quality Technology biased Impact of Open Access 17 initiatives

18 COMPARING WORLD RANKINGS (TOP 200) Country WEBR09 ARWU08 HEEACT08 QSTHES08 USA United Kingdom Germany 17 14* Canada Netherlands Japan Australia Switzerland France Sweden Italy Israel Belgium Denmark China & HK Korea Singapore Taiwan India Thailand

19 ACADEMIC DIGITAL DIVIDE 19

20 MOSTLY NATIONAL LINKS 20

21 IMPROVING THE RANK (WEB) Size Unique Domain More Research Distance Learning Visibility English Universities Backlinks Regional! Rich Files & Scholar Personal Pages Repository Hosting Third Parties 21

22 IMPROVING THE RANK (MISSIONS) Teaching Research Transfer More Technology Web 2.0 Distance Learning International Papers International Journals Community Engagement Local culture, society, economy 22

23 FUTURE DEVELOPMENTS JULY 2009 Reinforcing Ranking Mission Facing bad practices Deleting entries Combining domains Excluding supersites New visibility indicator Adapted G-factor Cooperation with Scimago World Ranking More info New regions Historical data Discipline Rankings By Department

24 LESSONS The Web is a more universal, democratic, cheaper and powerful communication tool and Web indicators could be used for wider, fairer and feasible evaluation purposes The Web Ranking provides useful information for developed and developing countries universities that it is not Web specific, measuring overall performance and global impact University web sites should reflect the full set of activities and all the results produced by the institution and help to their dissemination to improve visibility, prestige and attracting new students, excellent scholars and big funding Research based Rankings are measuring excellence, but not every University in the World is research intensive and focusing on these results could drive to scientific and cultural colonialism 24

25 THANK YOU! QUESTIONS? ISIDRO F. AGUILLO CYBERMETRICS LAB CCHS-CSIC ALBASANZ, MADRID SPAIN