Article-Level Metrics at PLoS & Beyond

Transcription

1 Article-Level Metrics at PLoS & Beyond Committed to making the world s scientific and medical literature a public resource Peter Binfield Publisher, PLoS pbinfield@plos.org

2 Where is the most appropriate place to evaluate a journal article? - At the Journal Level? - Or at the Article Level? And what are the most appropriate measures to use? - Citations? Usage? - Bookmarks? Tweets? - Wikipedia mentions? Blog coverage?

3 (

4 Article-Level Metrics (at PLoS) Article-Level Metrics (ALMs) at PLoS are not just about citations and usage. The concept refers to a whole range of measures which might provide insight into impact or reach ALMs are not simply about impact, they are also used for discovery and filtering We are providing metrics at the article-level, for every article, in every one of our titles We were the first publisher to provide this range of data, but others are now following

5 The ecosystem of research dissemination channels is broad and diverse Article usage Scholarly Citation Non- Scholarly Citations Blog Coverage Community Input News Coverage

6 We can capture these channels; present the data to users; generate new indications of impact; and develop new methods to filter and discover content Publication Dissemination ALMs Collectively as a suite, Article-Level Metrics aims to measure research impact in a transparent and comprehensive manner.

7 ALMs: Two Primary Use Cases Author Self-Metrics Researcher Research & Discovery Measure the impact and reach of their work Gauge value of any article with postpublication peer review Benchmark article performance against others Conduct custom searches that account for research impact Evaluate publication decisions to maximize the impact of their work Communicate impact of research to employers, funders, potential collaborators Other use cases for ALM data include: Librarian Research Funder Academic Institution Collect relevant articles and organize based upon array of criteria Gain insight into article s impact within context of related research Analyze trends/behaviors across a database of academic literature

8 How could we measure an article? Citations Web usage Expert Ratings Social bookmarking Social citations Community rating Media/blog coverage Commenting activity and more

9 How could we measure an article? Citations - Scopus, PubMedCentral, CrossRef, Web of Science Web usage - PLoS HTML, PDF, XML (COUNTER3) + PMC Expert Ratings - None yet (but F1000 is an example) Social bookmarking - CiteULike, Connotea, Mendeley Social citations - Facebook Likes, Twitter (coming soon) Community rating - user generated in 3 categories Media/blog coverage - ResearchBlogging, NatureBlogs, Bloglines Commenting activity - Notes and Comments and more

10 PLoS ALM Data Channels ALM Category Data Types and Channels Existing Article Usage HTML page views (COUNTER 3) PDF downloads (COUNTER 3) XML downloads (COUNTER 3) PMC HTML pageviews PMC HTML PDF downloads Additions (Bolded: near completion) Community Input (on the PLoS site) Citations - Scholarly Literature Reader Comments Reader Notes Ratings PubMed Central Scopus CrossRef Web of Science Microsoft Academic Search Citations - Non-Scholarly Literature Trackbacks Wikipedia Encyclopedia of Life F1000 Evaluations Media Coverage Trackbacks Google News (sunsetting) Yahoo News Blog Coverage Nature Blogs Google Blogs Research Blogging Postgenomic (discontinued) Bloglines Yahoo Search Reditt Social/Behavioral Mining CiteULike Connotea Facebook Likes Mendeley Readers/Groups Twitter tweets Stumble Upon bookmarks Zotero bookmarks Delicious bookmarks Digg bookmarks Diigo bookmarks Downloads to citation managers

11

12

13

14

15

16

17

18

19

20

21

22 Article-Level Metrics at PLoS Sloan Grant Build out ALMs (open source application with basic navigation and visualization tools) Enhanced Implementation on PLoS journals Advocacy and Outreach for adoption by researchers, funders & publishers ALM Technical Advisory Group Heather Piwowar & Jason Priem (Total Impact) Martin Fenner (Science Card) Dario Taraborelli (ReaderMeter & Wikipedia) Jevin West (EigenFactor) Johan Bollen (Mesur) Phil Bourne, Duncan Hull, Bjoern Brembs, Cameron Neylon, Pedro Beltrao (Open Science advocates & practicing academics)

23 New ALM Uses created by others With PLoS s open API, the uses of ALM data are unlimited. Examples of Third party re-use of the PLoS API: Total Impact ( Science Card ( Altmetric ( Total Impact Team Jason Priem / Heather Piwowar Mike Chelen

24 Science Card - Martin Fenner AltMetric / Impact Explorer, Euan Addie(Digital Science)

25 Issues to be aware of The landscape of data channels is constantly shifting Social Media outlets Media/ Blog outlets Bookmark channels New data sources emerge and old ones disappear Significance of each data channel constantly in flux (both waxes and wanes)??

26 Issues to be aware of Metric activity will depend on age of article Decay process of usage (HTML views) in six PLoS journals Yan K-K, Gerstein M, 2011 The Spread of Scientific Information: Insights from the Web Usage Statistics in PLoS Article-Level Metrics. PLoS ONE 6(5): e doi: /journal.pone

27 Issues to be aware of Dissemination activity also changes over time across data channels. Ex: Social Media Channels Bit.ly study of 1000 bitly links.

28 Issues to be aware of Activity in one metric can stimulate activity in other metrics Fig 8: altmetric activity timelines for the article doi: /journal.pone

29 Issues to be aware of Data channels have disparate degrees of significance Between channels Ex: 1 blog reference?= 1 Wikipedia reference 1 Tweet?= 1 Mendeley bookmark 1 Facebook Like?= 1 citation manager download 1 Facebook comment?= 1 PLoS article comment 1 NYTimes citation?= 1 New Scientist mention And within channels Ex: 1 blog reference?=? 1 blog reference

30 Issues to be aware of There is a wide variance in modes of research dissemination across fields Different researchers Different fields Different notions of impact - ALMs allow people to judge the value of many different metrics.

31 How Have ALMs been received? The PLoS article-level metrics are a substantial value-add for authors, including a range of download statistics, citations and social bookmarking data, and more. As an author, I would love to see this kind of service! Heather Morrison, Librarian, British Colombia Your innovation of the article level metrics is an extremely promising development in the evaluation of scientific publications. We are hopeful that this will transform the way that impact is assessed for published articles. Dr Michael Markham, Neurobiology, University of Texas at Austin (PLoS ONE Author) As paying customers of commercial publishers, should scientists and their funders be demanding more of this kind of information in the future? I reckon they should. Dr. Duncan Hull, European Bioinformatics Inst, UK (Blogger)

32 Downloading the data

33 Downloading the data (

34

35 Extending Article-Level Metrics Article level metrics are important, yes, but much more important for an Institutional Repository, is the ability to show Author level metrics, and Institution level metrics, David Palmer, Systems Librarian, Univ of Hong Kong

36

37

38 Alternative Metrics ( Alt-Metrics ) Priem, Taraborelli, Groth, Neylon

39 For Further Information Join our ALM list by mailing us at: Attend our ALM information session at the MLA, May 21 st ( ), with Donna Okubo. for more info The general PLoS ALM site is at : Altmetrics in the Wild preprint: Info on the broader alt metrics movement:

40 Peter Binfield Publisher, PLoS ONE e: