Management Metrics That Work Karen White USAID kwhite@usaid.gov Kris Vajs Federal Reserve Board Kris.M.Vajs@frb.gov Karen Krugman Export-Import Bank of the US Karen.krugman@exim.gov April 9, 2013
Agenda Advantages of Management Reporting Collecting, Storing, & Analyzing Metrics Presenting & Packaging Metrics Management Reporting at Our Organizations Questions & Answers
Why Report to Your Management? Reason 1 To communicate the library s value and demonstrate return-on-investment Reason 2 To generate support for continued or even increased staff and resources Reason 3 To demonstrate how the library supports the organization s mission and strategic goals Reason 4 To demonstrate transparency and accountability in how resources are used Reason 5 To show how library contributions have changed over time 3
Metrics: What Categories To Measure? Collect This Data: Library Size and Patron Base Transactions: Circulations and Reference Budget, Resource Allocation, and Value of Print & Digital Materials Facilities and Physical Foot Traffic Virtual Foot Traffic Education and Outreach Operations To Measure These Areas: Customer Satisfaction Return on Investment Effectiveness of Patron Outreach & Education Impact Comparability to Like Libraries 4
Sample Metrics Library Size and Patron Base Percentage of actual patrons to potential patrons Percentage of patrons seeking assistance in person, via e-mail, via phone, or via web Percentage of patrons by patron type Transactions: Circulations & Reference Total number of checkouts per year Percentage of checkouts by patron type Percentage inbound vs. outbound ILL requests Budget, Resources Allocation, & Value of Materials Percentage increase or decrease in overall budget over time Change in print vs. digital resource budget allocation over time Average cost per fulltext article downloaded See handout for more examples. 5
Sample Metrics (continued) Facilities & Physical Foot Traffic Virtual Foot Traffic Education & Outreach Operations Number of patrons entering library, year over year Average duration of a library visit Average duration of use of various physical spaces Average number of library website visits per day Most used pages on library website Increase in database users over time Number of database trainings and attendees per year Increase in database users after database trainings Number of new library users after attending orientation Average minutes to process, catalog, and shelve an item Average daily library staff time spent on core work vs. non-core work (troubleshooting computers, looking for passwords, etc.) See handout for more examples. 6
Storing Your Metrics Type Pros Cons Microsoft Excel No cost, easy to setup Can produce pie charts & graphs Can export to Access and create a database Inflexible Not a true database Relational Database (in-house or cloud-based) Virtual reference software (i.e., Altarama RefTracker, Springshare LibAnalytics) No cost, somewhat easy to setup Consistent data that can be searched, manipulated and reported Automatic calculations and reports Higher data storage capacity Data exportable to multiple tools Collects & analyzes data for you Reports & graphics on demand Consolidates requests from chat, email, phone & SMS text Supports a searchable knowledge base Ability to tag individual metrics Steeper learning curve than Excel. Needs care in planning and setup Sometimes created for a different purpose If in-house, needs to interact with the network on an ongoing basis. More expensive Security concerns if hosted by vendor Possibility of vendor going out of business 7
Collecting Your Metrics Decide what you re trying to measure before you begin to collect metrics Collect data proactively so you have them when needed Collect quantitative and qualitative metrics. Collect data, stories and testimonials Identify metrics from other libraries and organizations or industry standards that can be used to contextualize your metrics Look to library vendor-supplied usage data as a resource Limit the number of metrics to provide focus and to concentrate on the metrics with greatest impact 8
Ways To Collect Metrics Data Tally sheets/forms Surveys/Customer Feedback/Observations Focus Groups/Interviews Benchmarking 9
Interpreting Your Metrics Take simple measures Number of library patrons Use ratios to compare two measures Percentage of actual patrons to potential patrons Use trends to see changes over time Year-over-year increase or decrease in actual patrons to potential patrons Use benchmarks to compare to like libraries Combine benchmarks and trends to compare to like libraries over time Percentage of actual patrons to potential patrons vs. like libraries Year-over-year increase or decrease in actual patrons to potential patrons vs. like libraries See handout for more examples. 10
Presenting Your Metrics: Best Practices Report on a regular basis: weekly, monthly, quarterly or annually Use creative, single-use metrics to supplement your ongoing metrics program and illustrate unique points Tailor your reporting to different audiences (executives, your supervisor, the public, your customers) Compare metrics to past years, a small sample of similar libraries, or industry averages to create a frame of reference Use a mixture of quantitative & qualitative data; if possible, have your report tell a story with words, pictures, and numbers. Understand how your metrics relate to each other and be prepared to answer questions about what it all means Share metrics and reports with your staff to foster an understanding of how they contribute to the library s success 11
Presenting Your Metrics: Packaging Methods Written Reports Verbal Presentations Dashboards Infographics Impact stories 12
Our Management Metrics: USAID About the USAID Library Official Department Name Knowledge Services Center What the Organization Does Promotes economic prosperity; strengthens democracy; improves global health, food security, environmental sustainability & education; provides humanitarian assistance. Staff Size Subject Matter 11,000 Agency staff International development Our Customers USAID staff in Washington & overseas, U.S. government agencies, the public 13
Our Management Metrics: USAID Our Metrics Experience How We Collect the Data What Data We Collect How We Interpret & Report on the Data From our shared mailbox into SharePoint At the reference desk, saved in a Google Doc Customer feedback from emails into SharePoint Database usage from vendors saved in Excel Type of request Type of client (public, partner, USAID bureau or mission) Customized research by sector and topic Door count, circulation, PC & WiFi use E-resource usage Results of outreach activities (new employee orientations, office presentations, open houses) in terms of requests Use examples from customers Weekly, monthly & annual reports Mixture of quantitative & qualitative information Trends analysis How we contribute to the USAID Forward goals 14
Our Management Metrics: Ex-Im Bank About the Ex-Im Bank Library Official Department Name What the Organization Does Staff Size Subject Matter Our Customers The Research Library & Archives It backs the financing of US exports, mainly to emerging markets. Exports can be products (Boeing jets, oil rigs, nail files, candy) or services (engineering, architects) 400 FTE in agency; 7 staff in Research Library International Business and Economics Chair & Board, Economists, Bankers, Lawyers; Staff from Policy, Communications, Business Development 15
Our Management Metrics: Ex-Im Bank Our Metrics Experience How We Collect the Data What Data We Collect How We Interpret & Report on the Data Collected through Quickbase, a cloud-based relational database. Library staff report data via dropdowns, checkboxes, and written overviews of requests (mostly copied from e-mail) Anecdotal evidence collected through written and verbal follow-up Research, research, research: Who makes requests; purpose of requests; sources used; time spent; % actual vs. potential users; highest volume users To collect going forward: ROI of databases, gap analysis When we report: weekly reports detailing research requests; annual reports analyzing the year. Annual report contains different formats of information: charts showing trends, pie charts showing concentrations, tables showing impact, and bulleted lists detailing past accomplishments & future goals. 16
Our Management Metrics: Federal Reserve Board About the Board Research Library Official Department Name Board Research Library What the Organization Does Provide the nation with a safer, more flexible, and more stable monetary and financial system Staff Size Subject Matter Our Customers 1800 professional staff members; 13 staff in Research Library Economics, banking, and finance Economists, financial analysts, research assistants, and other professionals. The Library is open to all Board staff. 17
Our Management Metrics: Federal Reserve Board Our Metrics Experience How We Collect the Data What Data We Collect How We Interpret & Report on the Data Service Interaction Access Database used to collect and generate data on customer use of library services Ongoing metrics are supplemented by snapshot data demonstrating impacts. A snapshot looks at sample data to make a specific point SharePoint site money saved on data purchases SurveyMonkey just beginning to use this to get feedback from customers on projects and services Vendor statistics Access database customer name, division, library staff member, type of request, length of time to craft response, notes Our favorite metrics market penetration and percentage of repeat customers. We use repeat customers as an indication of customer satisfaction. ROI money saved and cost avoidance calculated for data acquisitions work. Annual budget presentation 18
More Resources To Get Started 5 Best Practices for Creating Effective Dashboards, Tableau White Paper, August 2011. http://www.tableausoftware.com/learn/whitepapers/5-best-practices-foreffective-dashboards Ard, Constance. Beyond Metrics: The Value of the Information Center, Information Outlook (September 2012). Ard, Constance. Adding Value to Corporate Libraries and Information Services. London: Ark Group, 2012. Davis, Hilary. Not Just Another Pretty Picture, 2009. http://www.inthelibrarywiththeleadpipe.org/2009/not-just-another-pretty-picture/ Dugan, Robert E., Peter Hernon, and Danuta A. Nitecki. Viewing library metrics from different perspectives: inputs, outputs, and outcomes. Santa Barbara, Calif.: Libraries Unlimited, 2009. Hales, Stuart. Metrics for Special Libraries, Information Outlook (September 2012). Hiller, Steve. Performance Measurement in Libraries, paper presented at the NISO Webinar on Measure, Assess, Improve, Repeat: Using Library Performance Metrics, September 8, 2010. http://www.niso.org/apps/group_public/download.php/4872/performance10web.p df 19
More Resources To Get Started Hiller, Steve. What Are We Measuring, and Does it Matter? Information Outlook (September 2012). Hiller, Steve. Performance Measurement in Libraries, paper presented at the NISO Webinar on Measure, Assess, Improve, Repeat: Using Library Performance Metrics, September 8, 2010. http://www.niso.org/apps/group_public/download.php/4872/performance10we b.pdf Kyrillidou, Martha. Looking Ahead: The Future of Performance Metrics, paper presented at the NISO Webinar on Measure, Assess, Improve, Repeat: Using Library Performance Metrics, September 8, 2010. http://www.niso.org/apps/group_public/download.php/4872/performance10we b.pdf More Alike than We Think, Library Journal vol. 136 no. 8 (2011). Stevens, Kerrie, Introduction to Library Metrics: Statistics, Evaluation, and Assesment, Australian and New Zealand Theological Library Association, unknown date. http://www.anztla.org/conference10/postconf10/stevens_1.pdf Strouse, Roger. Information management under fire: measuring ROI for enterprise libraries. Outsell, Inc., Nov. 9, 2007. 11 p. Strouse, Roger. Using ROI to support the IM value proposition. Outsell, Inc., May 27, 2010. 18 p. 20