Using research analytical tools with journal article. research leaders and programs

Transcription

1 Using research analytical tools with journal article databases and social media data to identify high impact impact research leaders and programs Elizabeth Brown Scholarly Communications and Library Grants Officer Binghamton University Libraries March 28, 2012 Spring 2012 ACS National Meeting Division of Chemical Information (CINF) Libraries and Institutional Research Evaluation

2 Knowledge is always evolving evolving Source: Map Maker: Guilllaume De L'Isle; Title: L'Amerique Septentrionale Dressee sur les Observations de Mrs. de l'academie Royale des Sciences... Rue des Canettes prez de St. Sulspice Avec Privilege du Roy pur 20. ans 1700; Place: Paris / 1700URL:

3 Islands of analytical data sources Library journal indexes/databases Article citations, references and co authors; Subject coverage Research analytical tools l t l Program, research and individual researcher/faculty strengths Informal scholarship Readership, contacts, outreach and discussion of work

4 Recent Projects at Binghamton SciVal Spotlight subscription ( ) Review of Web of Science and Scopus coverage (2011) Library peer analysis (2012 date) Social Media metrics and scholarly communications speakers/practitioners (2011 date): Where are the i Wh h disruptors?

5 Data Collection Levels Individual Academic Department University/Institution Academic Discipline/Program Not all tools collect data for all of these groups!

6 Who needs this information? Researchers/Faculty members Librarians Publishers/Library Vendors Higher Education Administration Social, Technology, News media companies Non profit academic groups Federal and other funding agencies

7 Source: Antique Maps of the World, Joanne Hevel (c.1696), Changhua Coast Conservation Action, Flickr, January 7, URL: (CC at nc sa)

8 What they collect Article citations and references Subject coverage, terminology Author, co author addresses, department information Grant funding information (sometimes) Some alternative metrics

9 Web of Science e ce Source: Web of Science, March 2012

10 Scopus Data Source: Scopus, January 2011.

11 Scifinder Scholar & Reaxys Very limited citation count sorting (500 references or less) Analyze by name, Get cited documents institution Source: Scifinder Scholar, March 2012

12 SciImago (University of Granada) Source: SciImago web site, March URL:

13 Reviewing WOS and Scopus (2011) Scopus content more STEM focused more conference proceedings less robust data (more later ) Wb Web of Science Better social science coverage Can measure people, departments, subjects Limited capacity for institutional data analysis

14 Source: Made in China, stvcr, Flckr, December 11, URL: (CC sa by)

15 What they Collect Institutional demographic data Student performance data Library collections, service data Faculty productivity data Budget data Research funding data

16 How data can be used Assess productivity Benchmarks for awards, recognition, curricula, scholarship hi Assess faculty workload levels, promotion and tenure processes Identify project and research funding Identify aspirational and institutional peers

17 SciVal Spotlightgives Spotlight gives a visual image of research output visual image ofresearch output Identifies: Current campus research strengths Emerging research areas Interdisciplinary Research groups Unmet collections needs Potential collaborators and peers Uses: Clustered Scopus citation data to i i d create research profile groupings and summaries Collexis semantic technology purchased by Elsevier El Data Source: SciVal Spotlight Global Data ( ); Scopus citation data ( ) 17

18 Aggregate publication lists identify journals, proceedings and books to strategically build the Libraries collections Publication totals show activity from a small set of journals/publishers Ave #pub/5 Library Journal (Proceeding) years Availability Lecture Notes in Computer Science 57 Print Proceedings of SPIE The International Society for Optical Engineering 48 No Materials Research Society Symposium Proceedings 26 Print Alcoholism: Clinical and Experimental Research li i l i lr h 24 Print/Elec Proceedings Electronic Components and Technology Conference 24 No ASME International Mechanical Engineering Congress and Exposition, Proceedings 17 No Langmuir 16 Print/Elec Physical Review B Condensed Matter and Materials Physics 15 Print/Elec Journal of Experimental Psychology: Animal Behavior Processes 15 Print/Elec Developmental Psychobiology 13 Print/Elec 2006 IIE Annual Conference and Exhibition 13 No Pharmacology, Biochemistry and Behavior 13 Print/Elec Journal of Electronic Packaging, Transactions of the ASME g 12 Print/Elec th IEEE Intersociety Conference, ITherm Elec Data Sources: SciVal Spotlight Global Data ( ); Scopus Citation Data ( ); Library Catalog Holdings, 1/

19 Ranked listing of distinctive, established, productive research groups and teams Supporting data on the global and regional impact of their work Rank Authors Department(s) Research Area(s) # pubs # cites 1Wu N.E.; Klir G.J.; Zacks S. EE; SSIE; Math Signal Processing; Fuzzy Sets; Computer Aided Process Planning Spear N.E.; Miller R.R.; Arias C.M. Psych Psychopharmacology; Consciousness; Neurotoxicology Meng W.; Zhao H.; Gupta C. ; Comp Sci Data Mining Younis M.I.; Alsaleem F.M. ME Sensors & Actuators; Semiconducting Materials; Nanotechnology Fridrich J.; Filler T.; Kodovský J. EE Data Mining; Image Processing Yammarino F.J.; Dionne S.D.; Gooty J. SOM Leadership & Organizational Behavior; Strategic Management Zhong C.J.; Luo J.P.; Wang L. Chem Sensors & Actuators; Nanotechnology; Electrochemistry Fordham B.O.; Cingranelli D.L.; Clark D.H. Poli Sci International Conflict; Political Studies Zhou J.; Zhang Y.; Braiotta L.B SOM Financial Accounting Wilson D.S.; Dlugos S;Dlugos M.J. Biol, EVoS Molecular Ecology; Consciousness; Bioethics Long B.; Zhang Z.M. Comp Sci Machine Learning; Data Mining Data Sources: SciVal Spotlight Global Data ( ), Scopus Citation Data ( ) 19

20 Emerging research areas and smaller research groups are profiled Publication history helps predict anticipated future collections needs Quantity of research generally valued over quality (citations) Rank Authors Dept(s) Research Area(s) # pubs # cites 3Gibb B.E.; Lenzenweger M.F.; Benas J.S. Psych Behavioral Research Therapy; Eating Disorders; Sex Roles Ponomarev D.V.; Ghose K.; Farnam D. Comp Sci Fault Tolerant Computing; Heat Transfer; Integrated Circuit Design Zaslavsky T.; Bowlin G.; Bowlin C Math Discrete Applied Math Coles M.E.; Pietrefesa A.S. Psych Behavioral Research Therapy; Clinical Psychiatry; Affective Disorders Regan P.M.; Clark D.H.; Frank R.W. Poli Sci International Conflict Westgate C.R. t CR EE Antenna Deak T.; Blandino P.; Barnum C.J. Psych Psychopharmacology; Neuroscience; Neuroimmunology McDonald M.D.; Heller W.B.; Shvetsova O.V. Poli Sci Political Studies; Political Science Meng W. Comp Sci Data Mining; Fuzzy Sets; Database Design & Management Wu N.E.; Srihari K.; Ramakrishnan ; S. SSIE Computer Aided Process Planning; Automatic Control Data Sources: SciVal Spotlight Global Data ( ), Scopus Citation Data ( ) 20

21 SciVal Spotlight Representation in Scopus and SciVal competencies did not correlate to library budget program allocations lt t lib b ll ti SciVal Subject Categories % of Binghamton Collections Budget ( ) Biology 11% Earth Sciences 6% Engineering 8% Chemistry 9% Math + Physics 14% Computer Science 4.1% Social Studies 26% Brain Research 7% Health Sciences 5% Medical Specialties Humanities 10.5% Source: SciVal Spotlight Data, ( ), BU Libraries Collections Budget Data (2009)

22 SciVal Spotlight Faculty Authors Faculty rank, productivity and representation in the competencies showed impact of research groups p p A mature department with fewer A younger, growing academic program with junior faculty more active junior faculty Rank Avg Pub/Date Range Avg % of art in comp. Professor 57 78% Professor 16 34% Professor 30 25% Associate 40 24% Professor 37 21% Associate 29 17% Professor 13 8% Professor 15 2% Distinguished 21 0% Rank Avg Pub/Date Range Avg % of art in comp. Associate 34 90% Assistant 13 85% Distinguished 18 51% Distinguished 65 51% Research 12 43% Assistant 16 39% Distinguished Teaching 14 29% Associate 34 22% Associate 21 11% Associate 35 3% Professor 18 0% Sources: SciVal Spotlight ( ), Binghamton University Campus Directory and Department Web Site data, July 2011

23 InCites (Thomson Reuters) Web of Science data used to generate institutional and personal analytical data Source: Thomson Reuters, Supporting your Strategic Research Vision in New York (program announcement), September URL: science.thomsonreuters.com/ny_agenda/

24 Academic c Analytics aytcs Source: Copyright 2012, Academic Analytics, LLC

25 National Center for Education Statistics (NCES): Integrated Postsecondary Education Data System (IPEDS) Peer Analysis of Libraries is possible: Location ARL member? 1. Volumes held (June 30 rep. yr) 2. Titles Held 16. Total Library Materials 21. Total Library Expenditures 27. Total Staff (FTE) University of Manitoba Yes ,306, ,592,516 $8,405,158 $22,586, University of Waterloo Yes 2,364,347 1,842,827 $7,052,562 $15,476, Colorado State University Yes 2,385,266 2,348,841 $7,021,010 $14,264, Virginia Polytechnic Institute & Yes State University ,634, ,799,403 $8,428,572 $15,164, Washington State University Yes 2,450,409 1,883,552 $5,935,225 $13,943, SUNY at Binghamton 2,444,112 1,872,915 $5,049,039 $10,472, Northern Illinois University 2,100,623 2,100,623 $4,812,133 $11,249, Kansas State University 2,614,274 2,290,738 $6,511,674 $13,118, Boston College Yes 2,602,677 2,149,945 $9,529,516 $20,338, Purdue University Yes 2,566,521 2,183,032 $11,244,162 $24,201, Iowa State University Yes 2,626,074 1,655,685 $9,078,625 $17,132, Rice University Yes 2,667,111 2,149,601 $10,015,546 $15,767, Source: ACRLMetrics, IPEDs data integration, February 2012

26 National Research Council (NRC) Doctoral Program Rankings Report (2011): Binghamton s s Doctoral Program Strengths Which doctoral programs are strengths within SUNY? Impacted library collections budgets for new and existing programs affected by campus growth include: Collection Family Doctoral Program Avg pub/fac Avg. cites/pub National StrengthSUNY Rank Humanities/Fine Arts Art History N/A 2 (2) Humanities/Fine Arts Comparative Literature N/A 2 (3) Humanities/Fine Arts English 8.14 N/A yes3 (4) Humanities/Fine Arts i Philosophyh N/A yes3 (4) Science/Eng/Nursing Behavioral Neuroscience yes2/3 (4) Science/Eng/Nursing Biology/Integrated Biology/Integrated Biomedical Sciences (4) Science/Eng/Nursing Chemistry (4) Science/Eng/Nursing Computer Science 1.64 N/A 3 (4) Science/Eng/Nursing Earth Science (4) Science/Eng/Nursing Ecology and Evolutionary Biology (2) Science/Eng/Nursing Electrical and Computer Engineering (3) Science/Eng/Nursing Mathematics yes2 (4) Science/Eng/Nursing Mechanical Engineering yes3 (3) Science/Eng/Nursing Nursing (1) Science/Eng/Nursing Systems Science (2) Social Science Anthropology yes2 (4) Social Science Economics (4) Social Science History 12.6 N/A yes2 (4) Social Science Political Science yes1 (4) Social Science Sociology (4) Data Source: National Research Council Doctoral Program rankings (2011) URL: 26

27 How would Binghamton rank with existing SUNY Doctoral Center Peers? Ranked Category Binghamton Proj. ARL Rank (n=116) Albany Buffalo Stony Brook Library Volumes 2,444, ,293,745 4,029,865 2,637,270 Unique Library Volumes 1,872, ,492,130 2,558,204 1,376,686 Microforms 1,875, ,922,357 6,187,843 3,851,230 Expenditures Books $977, $706,526 $1,074,840 $189,034 Expenditures Current Journals $4,031, $4,201,389 $6,920,542 $5,893,269 Total Expenditures Library Materials $5,049, $5,193,274 $8,214,014 $6,634,380 Books Purchased (volumes) 12, ,475 20,059 5,903 Current Journals (total) 82, ,417 87,997 87,737 Current Journals Purchased (titles) 52, ,804 60,532 62,163 Total Items Loaned (ILL) 26, ,401 32,793 21,379 Total Items Borrowed (ILL) 15, ,325 26,530 12,304 Total Expenditures Salaries and Wages $4,571, $6,328,585 $10,467,039 $6,288,060 Total Staff (FTE) 87 * Other Operating Expenses Ope $837, $36,554 $1,338,010 0 $695,341 Total Library Expenditures $10,472, $11,597,750 $20,062,539 $13,640,247 Data Sources: ARL Statistics , ACRL Statistics URLS:

28 Peer Data Analysis some advice Know the source of the numbers The quality of the analysis is as important as the data Identifying patterns can take time Look for tools that collect data from multiple sources

29 Source: No Wall Maps or Globes?, Scott McLeod, Flckr, August 13, URL: (CC BY)

30 Academic profile sites are everywhere Social networks ResearchGate Epernicus Academia.edu LinkedIn Campus based systems VIVO Proquest Community of Scholars Department, School web sites Closed networks ACS Network Campus Learning Management System: Blackboard, etc. etc

31 What they measure Discussion levels Readership levels Extent of sharing among peers Relationships between peers

32 Are researchers using social media? researchers social Discipline % using social media tools Earth Sciences 95.0 Environmental Sciences lsi Physics 88.6 Pharmacology, Toxicology 87.5 Neuroscience 87.0 Life Sciences 84.2 Social Sciences 84.0 Mathematics, Computer Science 83.7 Chemistry, Chemical Engineering 82.2 Materials Science and Engineering 82.1 Arts and Humanities 79.2 Biological Sciences 78.3 Health Sciences 74.8 Business and Management 73.7 All disciplines 79.7 Source: Social media and research workflow, CIBER, University College London, December 14, URL: media report.pdf

33 Social media types Source: Social media and research workflow CIBER University College % Activity Source: Social media and research workflow, CIBER, University College London, December 14, URL: media report.pdf

34 Source: On Government Trail looking across to Wizard Island, Flickr, OSU Special Collections and Archives, July 1, URL: (Public Domain)

35 Some examples PLoS ONE Almetrics Total Impact Mendeley Microsoft Academic Search i S Google Scholar Citations Allow individuals to access more analytical data Use library bibliometric data principles More examples at: p Wolters, Paul and Rodrigo Costas, Users, Narcissism and Control tracking the impact of scholarly publications in the 21 st century, SURF Foundation, February URL: 20control.pdf

36 PLoS ONE Altmetrics t Source: PLoS website, March 2012, URL:

37 total impact Source: total impact website, March 2012, URL: impact.org/collection/bbreoc

38 Mendeley Source: Mendeley website, March 2012, URL: impact.org/collection/bbreoc

39 Microsoft Academic Search Analytical data comparisons are possible, plus author comparisons and conference advisory tools Source: Microsoft Academic Search website, March 2012, URL: &id2=5302&topdomainid=5

40 Google Scholar Citations Shl ti Source: Google Scholar Citations, March 2012, URL:

41 Where are the Disruptors? Using: Analytical research data Institutional Rankings Participation in social networks Non traditional conference presence and activity Determine: Who s driving change ii h Where are they located? Is this activity challenging or complementing the current ranking/academic culture of influence? Source: Social Disruption website, URL: are the disruptors in academe part 2/

42 Source: In the Maze, a Wild Area in the Western Part of the Park Hikers Must Navigate by Map and Compass, 05/1972, Flickr, The National Archive. URL: (Public Domain)

43 What do these newer tools have in common? Growing need for reputation management tools beyond administrative offices Integrating multiple sources/networks li l Recognizing the increasing influence of web and social network activity on research dissemination Use or need bibliographic library or vendor data

44 What s limiting exploration? Data privacy concerns Interoperability of systems, formats and data fields Lack of reliable, consistent author and institutional identifiers (ORCID) Some sources not available to all ll

45 How are libraries central to this work? Workflow organized around Standardized data fields and retrieval formats Authority Control (clean data) Culture of sharing interlibrary loan cataloging records course reserves

46 Sources Where is the library in the digital workflow of research? Research in the age of Social Media, Swets/Mendeley, URL: in the age of social media whitepaper Angela van Barneveld, Kimberly E. Arnold and John Campbell, Analytics in Higher Education: Establishing a y g g Common Language, ELI Paper 1: January 2012, URL: Using Bibliometrics: A Guide to Evaluating Research Performance with Citation Data, Thomson Reuters, URL: science.thomsonreuters.com/news/ / / Co Citation Analysis: The Methodology of SciVal Spotlight, Elsevier, June URL: library/white paper co citation analysis methodology scival spotlight i / lib / hit it ti l i th l i l tli Kristen Fisher Ratan, Metrics: The New Black?, PLoS, NFAIS URL: the new black. Jennifer Howard, Tracking Scholarly Influence Beyond the Impact Factor, Chronicle of Higher Education, February 28, URL: scholarly influence beyond the impact factor/35565 Advanced Computing Infrastructure: Vision and Strategic Plan, National Science Foundation Cyberinfrastructure for 21 st Century Science and Engineering (CIF21), February URL: Paul Wouters and Rodrigo Costas, Users, Narcissism and Control tracking the impact of scholarly publications in the 21 st Century, SURFfoundation, February URL: February 2012 URL: org/news/surf report report users narcissism and control/. Social Media and research workflow, CIBER, University College London, December 14, URL: media report.pdf.

47 Thank You Binghamton University Libraries (Scopus/WOS Review Task Force Members, Dean John M. Meador. Jr.) Binghamton University Graduate School (Dean Nancy Stamp) Binghamton University Research Division Mark Hahnel (founder, Science 3.0) Source: Two skiers atop the summit of Mt. Baker standing next to a U.S. flag, Flickr, UW Digital Collections, February 1, URL: (Public Domain)