Software Performance Evaluation for the Evergreen Integrated Library System

Software Performance Evaluation for the Evergreen Integrated Library System

Introduction The Massachusetts Library Network Cooperative (MassLNC) is seeking proposals for consulting services to conduct a software performance evaluation of the Evergreen Integrated Library System (ILS). Evergreen is an open-source integrated library system developed by the Georgia Public Library Service to support 252 public libraries in the Public Information Network for Electronic Services (PINES) consortium. PINES completely transitioned to Evergreen in September 2006. Evergreen is now used by approximately 1,300 libraries around the world. It offers the following functionality for libraries: a metadata search engine (e.g. author, title, subject); a transaction processing engine (e.g. checkin, checkout, place request); a web application; a fund management system. MassLNC is a collaborative project among three Massachusetts library consortia that are using separate instances of Evergreen. Central/Western Massachusetts Automated Resource Sharing, Inc. (C/W MARS), as fiscal agent for MassLNC, will be the contracting agency. MassLNC will be the primary contact for coordinating the project and for making final decisions. When conducting this evaluation, the consultant(s) is expected to work directly with the open-source community supporting Evergreen. The budget for the proposed work is $50,000. Project Description Evergreen is a highly-scalable system used by many large library consortia as well as individual libraries. The size of the libraries and consortia using Evergreen varies. Appendix A provides data from one particularly large Evergreen system, C/W MARS, based in Worcester, Massachusetts. MassLNC and others in the Evergreen community have identified 1) places in the system where performance is suboptimal and 2) questions they hope can be answered through a performance evaluation. The list is available on the Evergreen wiki at: http://evergreen-ils.org/dokuwiki/doku.php?id=dev:testing:performance_issues Goals The goals for the performance evaluation are as follows: 1) to work with the community to plan for the software performance evaluation, including, but not limited to, the prioritization of areas that will be evaluated.

2) to conduct a performance audit of the Evergreen infrastructure to identify bottlenecks that negatively impact performance. 3) to work with the community to identify solutions and their impact. It is not expected that the performance consultant will implement solutions to address performance issues. Any recommended solutions will be left in the hands of the Evergreen developer community to implement. A performance evaluation is required in the three following target areas. A more detailed overview of each of these areas can be found at http://evergreenils.org/dokuwiki/doku.php?id=eg_developer_overview. Staff Client The Evergreen Staff Client is currently an application built on the XULRunner platform. XULRunner versions are roughly equivalent to Firefox versions for rendering purposes. The staff client is a combination of local and remote XUL pages, JavaScript, and the Online Public Catalog (OPAC). This includes use of the Dojo Toolkit, version 1.3 (released in March 2009). The OPAC is processed server-side with the Template Toolkit Perl Module. Specific Issues/Questions Related to the Staff Client The developer community has identified the move of staff client interfaces away from remote XULRunner as an initial step in improving staff client performance. The staff client has exhibited problems with memory leaks, a problem that has gotten worse at some sites with the release of Evergreen 2.3 in September 2012. Work has already begun to address these problems (see https://bugs.launchpad.net/evergreen/+bug/1086458). There have also been suggestions that an upgrade of the Dojo toolkit or the total elimination of Dojo from the staff client may resolve memory leaks and/or improve staff client performance. Would an upgrade of the Dojo toolkit or the total elimination of Dojo from the staff client help resolve these problems? Would a move of staff client interfaces away from XULRunner improve staff client performance? The staff client currently sends and receives individual XMLHTTPRequests. Are there any performance advantages to moving to an architecture that can support streaming requests? Messaging Evergreen is built upon the Open Scalable Request Framework, or OpenSRF. OpenSRF provides communication between various Applications and the clients that wish to call them. Each application provides a service, and usually the applications are referred to as services because of this.

OpenSRF currently works by passing JSON messages over XMPP. A very basic view of how a message passes through the system is as follows: 1. A request for a service is sent to the router. 2. The router chooses a listener and sends the request on. 3. The listener passes the message onto a drone. 4. The drone acts on the message and sends the response back to the original requester. Specific Issues/Questions Related to Messaging How does XMPP compare in performance to other modern messaging frameworks? Database Evergreen uses PostegreSQL, both in database design and stored procedures. Most stored procedures are written in SQL, PL/pgSQL, or Perl. PostgreSQL databases are split into schemas, allowing for logical groupings of related information. In addition to PostgreSQL itself, one should also be aware of the Fieldmapper Interface Definition Language (IDL), which is one of the ways Evergreen is told how to interact with the database. Fieldmapper maps database fields to Evergreen objects. The Fieldmapper IDL, or just IDL for short, is the Fieldmapper configuration file (fm_idl.xml) which tells Evergreen about all of the important aspects of the database. It is comprised of a set of class nodes, each of which contains at minimum a list of fields. In addition, each class may contain links, permissions, and in the case of fully virtual classes a block of SQL that defines the data to be retrieved. An important note: For the most part, clients and services can only talk to database elements that are defined within the IDL. The notable exception is stored procedures, which are not listed in the IDL. Specific Issues/Questions Related to the Database In general, are there database queries that can be optimized in the areas where performance has been identified as a problem? End users expect OPAC searches to be fast. Are there ways to optimize the searching of library materials through the OPAC?. Related to OPAC searching, in past versions of Evergreen, sites were able to utilize settings stored in the database to improve the relevancy of results retrieved through the OPAC. As of Evergreen 2.0, many sites deactivated the settings in this table because searches became too slow (see https://bugs.launchpad.net/evergreen/+bug/844374). Is there a way to optimize searching so that tools like these can be re-implemented?

Description of Services Required Prospective consultants should submit proposals that outline the process for the performance evaluation. The proposals should cover the following: Planning Proposals should outline how the consultant will work with the Evergreen community to plan for the performance evaluation, including further identification and prioritization of areas that will be evaluated, as well as working with MassLNC to ensure that a proper testing environment is available. Performance Evaluation Consultants are invited to submit proposals for one, any, or all of the target areas identified above (staff client, messaging, database). Proposals should outline the consultant s methodology for measuring performance in the target area. Communication As part of the proposal, consultants should identify preferred communication methods both in the planning phases and during the actual evaluation. Please indicate how status updates will be delivered through the course of the project. Please note that direct communication with MassLNC should be limited to contractual and coordination details. Other communication should be done directly with the Evergreen community using the following channels: The #Evergreen IRC channel on the Freenode network - http://evergreenils.org/irc.php. The Evergreen Development Discussion List and General Mailing List - http://evergreen-ils.org/listserv.php. The Evergreen wiki - http://evergreen-ils.org/dokuwiki/doku.php?start Responding to this Request Please submit your proposal electronically in PDF, OpenOffice and/or Microsoft Office format. In responding to this request, the proposal should include the following components: Methodology: Identification of the target area(s) (i.e. staff client, messaging, database) for which you are submitting a proposal and a description of the methodology that will be used to conduct the performance evaluation. Test Environment: MassLNC will provide a test environment for the performance evaluation. Please provide any preliminary details about the test environment that will be required to support the performance evaluation.

Timetable: Please include an estimated timeframe for the project. Cost Proposal: Please include an estimated cost for the project, keeping in mind that the project budget range is $50,000 to perform an evaluation in all three target areas. Relevant Experience: Detail your or your firm s experience in the following areas: o o o Conducting software performance evaluations, particularly for systems that handle a high volume of transactions; Working with Free/Libre/Open Source Software (FLOSS) communities; and any other experience you believe is relevant to this project. References: Please provide three references. Timeline Please return the completed proposal to Kathy Lussier by 11:59 p.m. (EDT) Wednesday, April 24, 2013. Requests for clarification or additional information should be sent to Kathy Lussier by 11:59 (EDT) Friday, April 10, 2013. Responses to these requests will be posted on the MassLNC web site at www.masslnc.org by April 17, 2013. The schedule for the issue of this RFP is expected to be as follows. The timetable may be subject to change at the discretion of MassLNC. Milestone Deadline Release of Request for Proposals Monday, April 1, 2013 Questions from potential respondents for clarification or additional information 11:59 p.m. (EDT) Wednesday, April 10, 2013 Answers to questions submitted by potential respondents Wednesday, April 17, 2013 Proposals due 11:59 p.m. (EDT) Wednesday, April 24, 2013 Selection of consultant May 3, 2013 Anticipated commencement date of work May 13, 2013

MassLNC reserves the right to: A discontinue this procurement process without obligation or liability to any potential vendor; B accept or decline any or all proposals; C accept all or any part of any proposal, unless specifically stated otherwise in the proposal by the respondent; D accept other than the lowest priced proposal; E award more than one contract. Contact: Kathy Lussier, Project Coordinator Massachusetts Library Network Cooperative klussier@masslnc.org 508-343-0128

Appendix A Data from the C/W MARS Evergreen System Central/Western Massachusetts Automated Resource Sharing, Inc. (C/W MARS) is an example of a large-scale consortium using Evergreen. The intent of the below data is to give prospective consultants an idea of the database size and level of activity that Evergreen needs to support with minimal impact on performance. Database statistics as of 3/10/2013 Number of Library Systems: 149 Number of Individual Library Branches: 170 Number of Users: 1,133,825 Number of Bibliographic Records (a record representing a specific edition of a title): 2,251,590 Number of Copy Records (individual barcoded items associated with the bibliographic records): 8,597,202 Volume of Transactions during Busy Hours Tuesday, March 5, 3-4 PM Checkouts: 6,204 Checkins: 5,930 Hold Requests: 715 Total Checkouts, Checkins, and Hold Requests: 12,849 Wednesday, March 6, 3-4 PM Checkouts: 6,224 Checkins: 5,356 Hold Requests: 641 Total Checkouts, Checkins and Hold Requests: 12,221 Thursday, March 7, 3-4 PM Checkouts: 5,200 Checkins: 3,984 Hold Requests: 631 Total Checkouts, Checkins and Hold Requests: 9,815

Appendix B - Glossary Fieldmapper Interface Definition Language (IDL) - The Fieldmapper IDL, or just IDL for short, is the Fieldmapper configuration file (fm_idl.xml) which tells Evergreen about all of the important aspects of the database. It is comprised of a set of class nodes, each of which contains at minimum a list of fields. In addition, each class may contain links, permissions, and in the case of fully virtual classes a block of SQL that defines the data to be retrieved. Integrated Library System (ILS) - An integrated library system is a system for libraries to track items ordered and owned, bills paid, and users who have borrowed materials. Online Public Access Catalog (OPAC) - The OPAC is the web application the public uses to search the database to find materials owned by the library. In Evergreen, the OPAC is also embedded in the staff client. The OPAC within the staff client may display some information that is not seen when accessing the OPAC via a standard web browser. Open Scalable Request Framework (OpenSRF) - OpenSRF provides communication between various Applications and the clients that wish to call them. Each application provides a service, and usually the applications are referred to as services because of this. OpenSRF works by passing JSON messages over XMPP Staff Client - The staff client is the client used by library staff to perform functions like ordering library materials, adding library materials to the database, checking materials in and out to users, sharing materials with patrons at other libraries, finding library materials, managing user information, accepting payments from users, and creating/retrieving statistical reports.