How to Measure the Impact of Research

Transcription

1 How to Measure the Impact of Research Dr. Fintan Bracken Librarian, Research Services & Bibliometrics 3 rd March 2016

2 Workshop Target Objectives At the end of this workshop, you will be able to: Explain the term bibliometrics; Find out how many times an article has been cited; Calculate the h-index of a researcher.

3 What is Impact? The direct and indirect influence of research or its effect on an individual, a community, the development of policy, or the creation of a new product, service or technology. (SFI, 2013)

4 Impact Types (based on SFI, 2013) Impacts on Professional Services Academic Impacts Environmental Impacts Economic & Commercial Impacts Health Impacts Societal Impacts Impacts on Public Policy & Services

5 Bibliometrics Bibliometrics focuses on the quantitative analysis of scientific and scholarly publications, including patents. Statistical analysis of bibliographic data by counting citations Measures patterns of authorship, publication & the use of the literature. Assess research impact of individuals, groups, institutions Measures impact (based on citations) not the quality of the work

6 Why use Bibliometrics? Measure of the impact of your research Identify highest impact or top performing journals in a subject area Identify the top researchers in a subject area Learning about a subject area 06/05/2015 Maynooth University

7 Examples of Metrics Publication counts Scholarly Outputs Citation counts Outputs in top percentiles % Documents in Top 1% & Top 10% Journal Impact Factor ScImago Journal Rank Journal Normalised Citation Impact Field-Weighted Citation Impact h-index

8 Tools to Find Citations Web of Science Scopus Google Scholar (Publish or Perish)

9 Web of Science (

10 Scopus (

11 Field-Weighted Citation Impact Actual citation count relative to the expected world citation count. A score greater than 1 is above the global average.

12 Google Scholar Google Scholar covers more books, conference papers, technical reports, etc. than Web of Science & Scopus Coverage is uncontrolled

13 h-index The h-index is a simple metric to quantify the scientific output of an individual (Hirsch, 2005) A h-index of 10 means that of all of the papers written by this author, 10 papers have been cited at least 10 times each Measure of consistency for a researcher's publications Only meaningful when compared to others within the same discipline area & at the same career stage Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46),

14 Title Journal Title Year Agriculture, Effects of set-aside management on birds breeding ecosystems & in lowland Ireland environment Google Scholar Citations Breeding bird populations of Irish peatlands Bird Study The impact of farming on over-wintering bird populations The diversity of birds and butterflies in Irish lowland landscapes with special reference to the effects of set-aside management on birds in the breeding season Lowland bogs, fens and reedswamps The value of the Open Access Repository to the Marine Institute The potential use of online tools for scientific collaboration by biology researchers Tearmann PhD Thesis Bird Habitats in Ireland GLINT Aslib Journal of Information Management Measuring the value of e-resources An Leabharlann h-index = 3 The h-index is 3 as 3 publications have at least 3 citations each. In order to obtain a h-index of 4, the 4 th ranked publication (the PhD thesis) would need to receive 3 more citations.

15 Web of Science (

16 Scopus (

17 Google Scholar Researcher Profile

18 Publish or Perish Free software created by Anne-Wil Harzing Uses Google Scholar data (or Microsoft Academic Search data)

19 Issues & Limitations Sciences (quantitative) vs Humanities (qualitative) Citation patterns can differ greatly between disciplines Some disciplines don t publish as much in journals Only published literature included Citation tools don t index all research areas or publications

20 Key Points for Responsible Use of Bibliometrics Citation patterns can vary widely between research fields Need to compare like with like Don t just use one citation tool Coverage varies in content, depth, discipline. Citation tools don t index all publications or research areas. Data needs to be looked at in context. Use a variety of metrics, data & other qualitative information where appropriate. Also remember that citation does not have to be positive! Maynooth University

21 Exercises 1. Using the Web of Science Core Collection ( what is Stephen Hawking s h-index (Hint: search for Hawking S* in Basic Search under the field Author & then Create Citation Report)? 2. What is the average number of citations his articles have received? 3. What is his second highest impact article based on citations & how many citations has it received? 4. How many articles did he author/co-author in 2011?

22 Recent Articles Hicks, D., Wouters, P., Waltman, L., de Rijcke, S. and Rafols, I. (2015) 'The Leiden Manifesto for research metrics', Nature [online], 520, , available: [accessed 26 Aug 2015]. Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Jones, R., Kain, R., Kerridge, S., Thelwall, M., Tinkler, J., Viney, I., Wouters, P., Hill, J. and Johnson, B. (2015) The metric tide: Report of the independent review of the role of metrics in research assessment and management [online] available: [accessed 26 Aug 2015]. Maynooth University

23 Thank You Questions??? Dr. Fintan Bracken Bibliometrics LibGuide Research Services & Bibliometrics Librarian Room GL0-023 Ext: