2 nd International Workshop on University Web Rankings CCHS-CSIC, Madrid (Spain). April 21 st 2009 THE RANKING WEB NEW INDICATORS FOR NEW NEEDS Isidro F. Aguillo Cybermetrics Lab. CCHS-CSIC isidro.aguillo@cchs.csic.es
AGENDA Why Ranking? Why Web? Comparison with other Rankings Ranking Web Future Developments The Floor is Open 2
ABOUT US Spanish Research Council Largest public research body in Spain 130 institutes (mostly basic science) 20% of the Spanish scientific output 2 Nobel Prizes Cybermetrics Lab Research group at the Center of Humanities and Social Sciences Editors of ejournal Cybermetrics EU Projects involved Staff: 4 researchers, 1 technician 3
TO RANK OR NOT TO RANK Global market Explosive international student mobility Growing demand of information Prior to 2003 there were no World Universities Rankings Demise of national rankings Impact of Shanghai s ARWU Trend towards increasing complexity Not obvious results, multiple classifications, confusing figures Student oriented Web publication reached World audiences League tables are easy to understand Unexpected results, rectors & politicians unaware of the actual situation But Methodological problems Mostly research based Promoting ranking-improvement tactics 4
STUDENT MOBILITY Countries of origin Countries of destination USA UK Australia Japan Germany Canada France Total Total from Asia 372173 152020 145338 122538 95829 75441 45780 1416263 of which, from Eastern Asia 204593 70809 62189 109594 36567 41049 21866 666024 of which, from South-central Asia 100842 35684 31548 3840 13729 19629 3462 351929 of which, from South-eastern Asia 36495 23681 43851 8691 7749 4392 6925 169195 of which, from Western Asia 30244 21846 3864 412 35532 7278 12741 215249 Total from all countries, 2006 584817 330078 184710 130124 261363 148164 247510 2924679 Total from all countries, 2000 475169 222936 105764 66607 187033 94401 137085 1894792 Growth in internationalization of tertiary education (1975-2006) Source: Education at a Glance 2008: OECD 5
HOW TO RANK? Indicators a 1. faculty/student a 2. funding a 3. papers published a 4. citations received a 5. degrees offered a 6. prizes a 7. external reviews a 8. web size a 9. intl. faculty a 10. foreign students a 999. computers ratio WeightsRank=w1*a1+w2*a2+ w 3 3 +..+w 999 999 0 w i 1 w i =1 A methodologically sound approach Considering many indicators, not only outputs but also inputs based Applied a multivariate non parametric analysis to estimate w i 6
FEASIBILITY PROBLEMS No data available for most of the countries Especially in developing ones in Africa, Asia and Latin America Data not comparable due to the lack of standards Professor/researcher/student/international student Poor ability to discriminate Ratios with low numbers Nobel Prizes (few universities with 2 or more) Bibliometric biases Biomedicine Social & Humanities, Technology North America, Europe Rest of the World English Other languages 7
WHY THE WEB? The Web is already the main tool for scholarly communication The Web reaches and it is accessed from every country of the World The Web could reflect all the missions of the universities Research Teaching Third mission The Web provides easy access to relevant indicators Web visibility has proved its ranking capabilities Google PageRank 8
THE RANKING WEB Original aim Promote Open Access Initiatives Target: Web Publish or Perish Political agenda Reaching developing countries Including all the missions, not only research Methodology Search engines as intermediaries: Easy way to collect data Link (quantitative) analysis as a proxy of quality assessment Impact Factor based composite indicator Consequences Webometrics Rank as an indicator: High correlation with other Rankings Academic Digital Divide Web (naming) bad practices: Taking the Web seriously 9
AN ACADEMIC MODEL Activity (50%) Impact (50%) Size Rich files Scholar Visibility Number of Webpages 20% Number of documents 15% Number of papers 15% Number of external inlinks Virtual referendum 10
WEBOMETRICS RANKINGS Published since 2004, inspired by the Shanghai Ranking Top 6,000 universities from a Directory with 17,000 entries Top 2,000 research centers from a Directory with 7,000 entries Two editions (January & July) per year Only organizations worldwide with their own web domain or subdomain Based on web indicators (activity and impact) 11
12
SUMMARY REGION / countries Top100 Top200 Top500 Top1000 Total NORTH AMERICA 7 78 121 206 392 3552 USA 70 104 179 354 3343 EUROPE 54 16 60 223 412 4531 United Kingdom 6 12 35 68 235 Germany 1 17 49 67 384 Italy 1 1 13 35 200 France 1 16 47 603 Spain 4 27 42 218 ASIA 33 3 10 40 118 4303 Japan 2 3 10 38 670 China & HK 3 10 22 1156 Taiwan 1 2 10 23 158 Korea 1 4 12 402 LATIN AMERICA 35 2 3 13 36 2891 Brazil 1 2 9 16 1529 Mexico 1 1 2 6 354 OCEANIA 12 1 6 15 35 131 Australia 1 6 13 29 79 ARAB WORLD 22 2 2 496 AFRICA 37 1 5 321 WORLD 200 16225 13
EXPECTED RESULTS North America Europe MIT (1 st region & world), Stanford (2 nd ) and Harvard (3 rd ) Universities University of Toronto (24 th world) Cambridge (1 st region), ETH Zurich (2 nd ) and Oxford (3 rd ) Universities Asia Tokyo (52 nd ), Taiwan National (55 th ) and Peking (117 th ) Universities Oceania National University of Australia (48 th ) Latin America Universidad Nacional Autónoma de México (44 th ) Universidade de Sao Paulo (87 th ) Africa University of Cape City (359 th ) 14 WR (Enero 2009)
MORE RESULTS Bad naming practices Universities of Catalonia (.es,.edu.,.net,.cat) Old and new domains (Imperial, Illinois, Victoria, Northwestern, Case Western, Pontificia de Chile, ) Common domain Splitted domain Japanese, French, Italian Jussieu campus (Paris 6, Paris 7, CNRS) Strasbourg Universities (3 under common domain) University of Helsinki (shared with the City Hall) Johns Hopkins University Universidad de la República (Uruguay) University of Zagreb Warsaw University Websites mostly in local language (Few pages in English) Biases Against Biomedical small size organizations Favoring Technology oriented and Distance Learning 15 based universities
COMPARING WORLD RANKINGS CRITERIA WEBOMETRICS ARWU HEEACT QS-THES Spain (research lab) China (university) Taiwan (gov agency) United Kingdom (firm) Univs analysed Univs ranked 17000 6000 3000 500 3500 500 2000 500+ Teaching Alumni Nobel 10% Students/Staff 20% Internationalisation % Foreign Students 5% % Foreign Professors 5% Size Webpages 20% University 10% Research Documents 15% Nature&Science 20% Papers S&SSCI Papers GS 15% Papers S&SSCI 20% 20% Impact Links 50% Highly Cited Researchers 20% Prestige Faculty Nobel 20% Highly Cited Res Cit High IF Journals 50% Citations S&SSCI 30% Citations Scopus 20% Academic Survey 40% Employers Survey 10% 16
ADVANTAGES & DISADVANTAGES ARWU QS-THES HEEACT WEBOMETRICS Focus on excellence Perceived prestige Recent performance Developing countries Historical achievements Long term performance Other missions than research Estability Internationaliza tion Only research Normalization problems Easy to build Only research Bibliometric biases Discriminatory capability Prizes assignation University Hospitals Representation bias of the survey Size related Not stable Ties (beyond rank 200) Private company Size related problems No crown indicator Sensible to URL bad practices Google Scholar data quality Technology biased Impact of Open Access 17 initiatives
COMPARING WORLD RANKINGS (TOP 200) Country WEBR09 ARWU08 HEEACT08 QSTHES08 USA 104 90 91 58 United Kingdom 12 22 19 30 Germany 17 14* 18 11 Canada 17 6 9 12 Netherlands 6 9 9 10 Japan 3 9 9 10 Australia 6 6 6 9 Switzerland 3 6 7 7 France 1 7 4 4 Sweden 4 4 4 4 Italy 1 5 7 1 Israel 1 4 3 3 Belgium 1 4 2 5 Denmark 1 3 2 3 China & HK 3 0 3 10 Korea 1 1 1 3 Singapore 1 1 1 2 Taiwan 2 1 1 1 India 0 0 0 2 Thailand 0 0 0 1 18
ACADEMIC DIGITAL DIVIDE 19
MOSTLY NATIONAL LINKS 20
IMPROVING THE RANK (WEB) Size Unique Domain More Research Distance Learning Visibility English Universities Backlinks Regional! Rich Files & Scholar Personal Pages Repository Hosting Third Parties 21
IMPROVING THE RANK (MISSIONS) Teaching Research Transfer More Technology Web 2.0 Distance Learning International Papers International Journals Community Engagement Local culture, society, economy 22
FUTURE DEVELOPMENTS JULY 2009 Reinforcing Ranking Mission Facing bad practices Deleting entries Combining domains Excluding supersites New visibility indicator Adapted G-factor Cooperation with Scimago World Ranking More info New regions Historical data Discipline Rankings By Department 2010..2012 23
LESSONS The Web is a more universal, democratic, cheaper and powerful communication tool and Web indicators could be used for wider, fairer and feasible evaluation purposes The Web Ranking provides useful information for developed and developing countries universities that it is not Web specific, measuring overall performance and global impact University web sites should reflect the full set of activities and all the results produced by the institution and help to their dissemination to improve visibility, prestige and attracting new students, excellent scholars and big funding Research based Rankings are measuring excellence, but not every University in the World is research intensive and focusing on these results could drive to scientific and cultural colonialism 24
THANK YOU! QUESTIONS? ISIDRO F. AGUILLO CYBERMETRICS LAB CCHS-CSIC ALBASANZ, 26-28 28037 MADRID SPAIN E-MAIL: ISIDRO.AGUILLO@CCHS.CSIC.ES WWW.WEBOMETRICS.INFO 25