1 Using analytics to inform library practice Joy Palmer, Mimas This work is licensed under a Creative Commons Attribution 3.0 Unported License
2 Why do this?
3 Library Data at Huddersfield
4 Library Impact Data Project
5 To support the hypothesis that There is a statistically significant correlation across a number of universities between library activity data and student attainment
6 Final grade achieved Number of books borrowed Number of times e-resources were accessed Data used Number of times each student entered the library, e.g. via a turnstile system that requires identity card access School/Faculty
7 Library Impact Data Project 1 Showed a statistical significance between: Final grade achieved Number of books borrowed Number of times e-resources were accessed Across all 8 partners
8 Library Impact Data Project Phase I looked at over 33,000 students across 8 universities Phase II looked at around 2,000 FT undergraduate students at Huddersfield
9 Now with additional data: Demographics Discipline Retention On/off campus use Breadth and depth of e-resource usage UCAS points (entry data) Correlations for Phase 1 Library Impact Data Project 2
10 Conclusions: Library Impact Data Project 2 We showed statistical significance for demographics such as age, gender, ethnicity and country of origin We showed statistical significance across top level subjects and within these disciplines We showed a connection between library use and retention We showed the depth and breadth of a collection may make a difference
11 Drivers for change 1 And we know all this is firmly on Libraries radars Our survey: How important will analytics be to academic libraries now and in the future, and what is the potential for a service in this area?
12 What about sharing your data about usage with other institutions? There s a significant appetite for analytics services.but hesitation over sharing entry data and other student data than other forms of usage data. Only 46% would be willing to share data if the institution was named. But if institutional identity can be anonymised, that changes to 91%
13 And is this a current strategic priority?
14 What about in the next five years?
16 Can we collect data from institutions and create tools that allow libraries to analyze how their resources are being used, when and by whom?
17 What about the benefits of scale? Can this dashboard also give institutions the tools to compare or even benchmark usage against other institutions?
18 What data can we use and get a hold of? UCAS data, loan data, eresource logins.. (but not data on usage of individual items) (yet)
19 Our collaborators and data providers:
20 data wrangling: Getting, analyzing, cleaning, and presenting that data
22 Working with the API to present the data How should users work with the data? What do they want to be able to do? What do they do? What does the system do?
23 The Epic User Stories connect the library with the university mission contribute to the institutional analytics effort demonstrate value added to users ensure value from major investments develop investment business cases impact student measures of satisfaction, such as NSS address measures of equality and diversity of opportunity inform / justify library policy and decisions as evidence led engage stakeholders in productive dialogue identify basket of measures covering all key areas inform librarian professional development enable the sector to understand the questions to be answered
26 Job stories
27 JiscLAMP What did we achieve? LAMP project outputs We managed to clean up and process the data from all of the partners We created a prototype our analytics engine We performed a benchmarking exercise We showed that the idea of a shared library analytics service was feasible
28 » We can demonstrate usage by cohorts: Department Degree name Course Course type? Gender/Ethnicity/Nationality/Disability/Age Level of attainment Attendance mode (full time/part time) UCAS points What can we do with the data?
29 What can we do with the data? And we can demonstrate correlations between usage and attainment, usage and cohort (and attainment and cohort)
33 And we can potentially signal if findings are statistically significant or not
34 A brief (important) word on ethics Should we be holding and analyzing this kind of data? Data protection issues & Big brother concerns All students pay the same fees shouldn t they be treated the same? But what if we didn t do this What would the reaction be if it was found that we had this data but didn t act on it? We have a duty to care for the individual wellbeing of our students
35 But where exactly does the user journey and workflow begin? Within the system or outside of it?
36 How much do we assume the user is analysing the data? And much analysis should the tool perform on behalf of the end user?
37 » Here s a simple question: How do humanities and social science students use books? * * All the data used in this presentation is completely made up. Any resemblance to real university library usage data, living or dead, is purely coincidental.
38 » And a simple answer Humanities bigger users than social sciences
39 » But wait! Humanities still bigger users, but the difference isn t so stark
40 » What about other factors? In both disciplines, full-time students borrow more books than part-time ones The difference is smaller for social scientists than for humanities students Part-time students seem quite similar across disciplines
41 » Who s not borrowing? So although the means were fairly similar (2.9 to 2.3), proportionally there are lots more social scientists who have never borrowed a book.
42 » Who s not borrowing? There s clearly a big problem with part-time students in the humanities In the social sciences, everyone is equally in need of help Social science part-timers are more likely to borrow books than humanities part-timers why might that be?
43 » Who s not borrowing? Shows median borrowing (humanities = 3, social sciences = 2) Shows upper and lower quartiles Shows max and min values Standard way of describing data but is it useful here?
44 Where do we go from here? Phase 2» We have funding for Phase 2» We re now testing the ugly prototype and developing user facing wireframes» Currently putting together the new project and marcomms
45 Our key areas for focus » Make the data beautiful and compelling. Develop a dashboard UI through iterative testing and development
47 Our key areas for focus » Usage data to profile individuals, e.g. for REF or intervention purposes? What are the ethical or legal issues?
48 Our key areas for focus » eresource item level usage and the current approach of the UK Access Management Federation Is it possible to crack that nut?
49 Our key areas for focus » NSS data and SCONUL stats. Integration would be of major value. How can we bring that data into scope?
50 Our key areas for focus » Data literacy What does it mean? Who needs it? What needs to be automated and what needs to be taught as a skillset?
51 Our key areas for focus » Benchmarking. The killer app? Is there a business case for the service if it doesn t provide the capability to compare across institutions? How would this work?
52 JiscLAMP Phase 2 Key contacts and partners for next phase HESA (NSS) Shibboleth/Athens SCONUL Performance Group Developing the business case and model
53 How can you get involved?» Follow and comment on our blog: Volunteer to take part in user testing and focus groups» Become a data contributor!
THE CHILD AT THE CENTRE SELF-EVALUATION IN THE EARLY YEARS THE CHILD AT THE CENTRE SELF-EVALUATION IN THE EARLY YEARS Crown copyright 2007 ISBN: 978-0-7053-1109-0 HM Inspectorate of Education Denholm House
Students experiences of full-time Foundation degrees A Report to fdf Holly Higgins, Jane Artess and Isla Johnstone 1 Students experiences of full-time Foundation degrees Contents Foreword 3 Executive Summary
Moving towards an open access future: the role of academic libraries Siân Harris A report on a roundtable commissioned by SAGE, in association with the British Library August 2012 www.sagepublications.com
Climate Surveys: Useful Tools to Help Colleges and Universities in Their Efforts to Reduce and Prevent Sexual Assault Why are we releasing information about climate surveys? Sexual assault is a significant
HIGHER EDUCATION Students at the Heart of the System JUNE 2011 Higher Education: Students at the Heart of the System Presented to Parliament by the Secretary of State for Business, Innovation and Skills
HOW GOOD ARE WE NOW? HOW GOOD IS OUR SCHOOL? HOW GOOD CAN WE BE? THE JOURNEY TO EXCELLENCE: PART 3 HOW GOOD ARE WE NOW? HOW GOOD IS OUR SCHOOL? HOW GOOD CAN WE BE? THE JOURNEY TO EXCELLENCE: PART 3 Please
E-journals: their use, value and impact final report E-journals: their use, value and impact final report A Research Information Network report January 2011 www.rin.ac.uk This report is the second arising
Julie Arendt, Megan Lotts 155 What Liaisons Say about Themselves and What Faculty Say about Their Liaisons, a U.S. Survey Julie Arendt, Megan Lotts abstract: Liaison librarians and faculty in chemistry,
Communicating and Using Climate Survey Results Climate Studies can provide helpful information about different indicators of a school s campus climate. This information can be used to educate the campus
Clinical Trials: What You Need to Know Clinical trials are studies in which people volunteer to test new drugs or devices. Doctors use clinical trials to learn whether a new treatment works and is safe
Innovation Roles The People You Need for Successful Innovation A White Paper By Dean Hering Jeffrey Phillips NetCentrics Corporation November 1, 2005 NetCentrics 2005. All rights reserved. 1 Table of Contents
Defining Materiality: What Matters to Reporters and Investors Do investors and reporters agree on what s material in the Technology Hardware & Equipment and Banks & Diverse Financials sectors? ABOUT GRI
MISSION AND GOALS 34 3 DEVELOP A MISSION, AND GOALS When organizations or individuals decide to join forces and work as a partnership, they do so with a purpose in mind. While the individual partners may
Understanding the process to develop a Model of Care An ACI Framework A practical guide on how to develop a Model of Care at the Agency for Clinical Innovation. Version 1.0, May 2013 AGENCY FOR CLINICAL
REPORT General counsel: vague about value? A survey and discussion paper Contents 01 Foreword 02 Flashback: The GC value pyramid 05 Key findings 06 Adding value 08 Case study: Daragh Fagan, General Counsel
Securing Jobs for Your Future Skills for Victoria MESSAGE FROM THE PREMIER AND THE MINISTER FOR SKILLS AND WORKFORCE PARTICIPATION Victoria s unprecedented prosperity over the past decade has been created
Making Smart IT Choices Understanding Value and Risk in Government IT Investments Sharon S. Dawes Theresa A. Pardo Stephanie Simon Anthony M. Cresswell Mark F. LaVigne David F. Andersen Peter A. Bloniarz
Teachers Know Best Making Data Work For Teachers and Students ABOUT THIS STUDY As part of the Bill & Melinda Gates Foundation s broader efforts to improve educational opportunities for all students, the
How to Develop a Logic Model for Districtwide Family Engagement Strategies Helen Westmoreland, Consultant M. Elena Lopez, Senior Consultant Heidi Rosenberg, Senior Research Analyst November, 2009 For questions
OVERVIEW Brief description This toolkit deals with the nuts and bolts (the basics) of setting up and using a monitoring and evaluation system for a project or an organisation. It clarifies what monitoring
Volume 9, 2006 Approved April 2006 ISSN: 1523-4320 www.ala.org/aasl/slr Flexible Scheduling: Implementing an Innovation Joy McGregor, Senior Lecturer and Course Coordinator of Master of Education (Teacher
Turning Strategies into Action Economic Development Planning Guide for Communities Turning Strategies into Action Table of Contents Introduction 2 Steps to taking action 6 1. Identify a leader to drive
building a performance measurement system USING DATA TO ACCELERATE SOCIAL IMPACT by Andrew Wolk, Anand Dholakia, and Kelley Kreitz A Root Cause How-to Guide ABOUT THE AUTHORS Andrew Wolk Widely recognized
The Work-related Learning Guide First edition A guidance document for employers, schools, colleges, students and their parents and carers The Work-related Learning Guide The Work-related Learning Guide
Is Connectivity A Human Right? For almost ten years, Facebook has been on a mission to make the world more open and connected. For us, that means the entire world not just the richest, most developed countries.
March 2014/02 Research This report analyses trends in continuation rates in English higher education, and uses statistical modelling techniques to isolate the effect of institutional bursaries on retention.
Research report January 2010 CREATING AN ENGAGED WORKFORCE CREATING AN ENGAGED WORKFORCE FINDINGS FROM THE KINGSTON EMPLOYEE ENGAGEMENT CONSORTIUM PROJECT This report has been written by: Kerstin Alfes,
March 2014 Making time for great teaching Dr Ben Jensen Grattan Institute Support Grattan Institute Report No. 2014-3, MARCH 2014 Founding members Program support Higher Education Program This report was
Talking about good professional practice Views on what it means to be a good doctor Talking about good professional practice 2 Table of Contents Acknowledgements 2 FOREWORD 3 INTRODUCTION 4 EXECUTIVE SUMMARY