Evaluation of the impact of library discovery technologies on usage of academic content Presenters: Angela Conyers, Evidence Base (BCU) Valérie Spezi, LISU (Loughborough University)
Why this study? Commissioned by UKSG/Jisc in July 2013 Lots of interest in library discovery technologies Questions about whether libraries, publishers and other stakeholders should be engaging with those technologies Follows on from the UKSG research report on Link resolvers and the serials supply chain and the work of KBART Small-scale study A UK perspective No previous usage data analyses fill in the gap 2 studies about to report shortly Work started in July 2013
Objectives of the research Evaluation of the impact of library discovery technologies on usage of academic resources Provide evidence to determine if there is a case for Investment in library discovery technologies by libraries Engagement with library discovery technologies by publishers and other stakeholders in the information supply chain Provide recommendations for stakeholders to best support the discovery of academic resources Identify additional research, data, discussion and initiatives that will support the findings of the study
Methodology Phase 1: survey of UK HE libraries Objective: determine the current landscape Phase 2: case studies of libraries and publishers Objective: collect usage data + views and perceptions on the impact of library discovery technologies Phase 3: interviews with stakeholders Objective: obtain a bigger picture on the perceived impact of library discovery technologies and an insight of where the sector is going
Methodology (cont d) Survey: 62 respondents Online questionnaire distributed to UK HE library directors 8 publishers and stakeholders 6 case study libraries Data received from 6 libraries & 4 publishers/content providers COUNTER JR1, BR2 and DB1 or close equivalent
Phase 1: UK landscape use in academic libraries The UK HE landscape: No of respondents 25 20 15 10 5 0 In process Implemented 21 10 7 5 4 2 1 1 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2 77% of UK HE libraries are already using an at their institution A further 11% are in the process of implementing an implementation in HE libraries had probably reached its peak in the last 12 months
Phase 1: UK landscape products used by UK HE libraries AquaBrowser 2% 2% 2% 1% 2% Blacklight Ebsco Discovery Service Encore 3 products dominate the library discovery market in our sample 36% 26% 24% 2% 3% Endeca Primo Summon VuFind WorldCat Local Half of the survey respondents considered the to be a replacement for their previous OPAC, although there was an indication that online catalogues are still needed for some specific transactions Other
Phase 2: usage study What are we going to present? 1. General findings general picture Data received from the 6 participating libraries and 4 of the participating publishers Libraries usage data journal, e-book and database Publishers usage data journal 2. Findings for a case study library more detailed picture
VS2 Overall journal usage trends 6.00 5.00 implementation E 4.00 Relative usage per FTE 3.00 2.00 1.00 0.00 B D F A C -1.00-2.00-12 - 10-8 - 6-4 - 2 + 2 1st year + 4 + 6 + 8 + 10 + 12 + 14 + 16 2 nd year + 18 + 20 + 22 + 24
Slide 9 VS2 there may be a new graph - see Claire Valérie, 05/11/2013
Overall journal usage trends constant titles 5.00 4.00 implementation E A C 3.00 2.00 1.00 0.00 F B D -1.00-2.00 + 24-12 - 11-10 - 9-8 - 7-6 - 5-4 - 3-2 - 1 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11 + 12 + 13 + 14 + 15 + 16 + 17 + 18 + 19 + 20 + 21 + 22 + 23 Relative usage per FTE
Overall e-book usage trends 15.00 13.00 11.00 9.00 F D B 7.00 5.00 3.00 1.00-1.00-3.00 E A + 15 + 16 + 17 + 18 + 19 + 20 + 21 + 22 + 23 + 24 Relative usage per FTE - 12-11 - 10-9 - 8-7 - 6-5 - 4-3 - 2-1 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11 + 12 + 13 + 14 implementation
Overall e-book usage trends constant titles 13.00 11.00 implementation A 9.00 E 7.00 F 5.00 3.00 D 1.00 B -1.00-3.00 Relative usage per FTE - 12-11 - 10-9 - 8-7 - 6-5 - 4-3 - 2-1 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11 + 12 + 13 + 14 + 15 + 16 + 17 + 18 + 19 + 20 + 21 + 22 + 23 + 24
Overall database usage trends 3.50 2.50 B 1.50 0.50 C F -0.50-1.50-2.50-3.50-4.50 D -5.50 + 15 + 16 + 17 + 18 + 19 + 20 + 21 + 22 + 23 + 24 Relative usage per FTE - 12-11 - 10-9 - 8-7 - 6-5 - 4-3 - 2-1 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11 + 12 + 13 + 14 VS3 implementation
Slide 13 VS3 check graph again with claire - ok for one library omitted but what about the other one missing? Valérie, 05/11/2013
Publishers - overall journal trends 0.40 W 0.30 implementation 0.20 0.10 Z Y X 0.00-0.10-0.20-0.30 + 16 + 17 + 18 + 19 + 20 + 21 + 22 + 23 + 24 Relative usage per FTE - 12-11 - 10-9 - 8-7 - 6-5 - 4-3 - 2-1 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11 + 12 + 13 + 14 + 15
Broad picture key findings What does the usage analysis suggest? Journals some impact E-books definitely a positive impact Databases cannot tell Interpretation of the aggregated data challenge - why? Multi-dimensional environment lots of noise in the results other factors may affect usage, notably the volume of content available We ve tried to control for this with the constant titles but there may be other factors No common pattern by type of resources for each library Except for e-books maybe? Next: look at a case study library in greater detail
Overall time series for journals, ebooks and databases data for a single case study library 100,000,000 10,000,000 1,000,000 100,000 10,000 Sep-08 Nov-08 Jan-09 Mar-09 May-09 Jul-09 Sep-09 Nov-09 Jan-10 Mar-10 May-10 Jul-10 Sep-10 Nov-10 Jan-11 Mar-11 May-11 Jul-11 Sep-11 Nov-11 Jan-12 Mar-12 May-12 Monthly usage total Jul-12 Sep-12 implemented JR1 BR2 DB1 JR1 trend BR2 trend DB1 trend Logarithmic scale
Journal usage per FTE for library E Relative usage per FTE 6 5 4 3 2 1 0-1 -2-12 - 10-8 - 6 All Constant titles W X Y Z - 4-2 + 2 + 4 Increased journal title availability + 6 + 8 + 10+ 12+ 14+ 16+ 18+ 20+ 22+ 24 Journal usage levels increasing before effect on constant titles shows a decrease in usage level immediately after implementation and then a sustained increase at a higher pace
E-book usage per FTE for library E 16.00 14.00 All Overall usage going up Relative usage per FTE 12.00 10.00 8.00 6.00 4.00 2.00 0.00-2.00-12 - 10-8 - 6-4 - 2 + 2 + 4 + 6 New e-book collections + 8 Constant titles W + 10+ 12+ 14+ 16+ 18+ 20+ 22+ 24 Constant titles going up Increase inflated by one Subscription making up for 70% of the usage of the constant title set throughout E-book usage for publisher W increased by a factor of 8
Database usage per FTE for library E 250 Constant titles Relative usage per FTE 200 150 100 50 0 Suspected multiple counting issue excl. platform M excl. resource N database usage increased significantly immediately after Issues with database counting? -50-12 - 10-8 - 6-4 - 2 + 2 + 4 + 6 + 8 + 10+ 12 + 14 + 16 + 18+ 20 + 22 + 24
Usage analysis - summary What can we say from the usage analysis? No straightforward answer but it seems that overall library discovery technologies influence positively the use of academic resources The effect may vary according to the type of resources e-books seem to benefit greatly from Isolating the sole impact of is a challenge multi-dimensional environment Many other factors may affect usage we have tried to control for content growth by providing an analysis including constant titles More data needed for a meaningful analysis 2-year post-implementation data is not enough to pick up a trend and isolate other variables influencing usage
Additional findings from libraries Perceived advantages for libraries: Generally, high levels of library satisfaction with Increased usage borne out by usage data Enhanced user experience (primary motivation not increased usage) One stop shop = single interface linked to full text Better use of subscriptions no silos Perceived challenges: Usage data analysis not done routinely searching aimed at undergrads? Starting point? Can researchers benefit from too? Lack of clarity in coverage from suppliers Interoperability between systems Lack of co-operation between some publishers and some suppliers
Additional findings from publishers and content providers Perceptions when engaging with : Improving discoverability and visibility of content still very low traffic from compared to search engines particularly relevant for smaller publishers? better service for their authors and readers can publishers afford to wait and see where it is going? Perceived challenges: Metadata optimisation for improved discoverability Dilution of the publisher s brand within the Lack of feedback/communication from suppliers Lack of visibility and understanding of how data are used Relevancy ranking
Other stakeholders in the information supply chain some key findings great tool probably changing the library landscape in the short to medium term But the use of raises some questions: Are a long term valid solution to Web-scale information searching? Costs & benefits to libraries? What is the effect on A&I databases for specialist information?
THANK YOU! Recommendations soon available in the full report that will be presented to UKSG For a copy of the report: Check out the UKSG website - http://www.uksg.org/ Contact us for a copy of the report - lisu@lboro.ac.uk