The PIRUS Code of Practice for recording and reporting usage at the individual article level

PIRUS A COUNTER Standard The PIRUS Code of Practice for recording and reporting usage at the individual article level (Note: this draft is for public consultation only and is not yet ready for full implementation) Release 1 Published November 2012 Draft 1 Abstract The PIRUS Code of Practice provides specifications for the recording and reporting of usage at the individual article level that are based on and are consistent with the COUNTER Code of Practice for e-resources. The PIRUS Code of Practice provides the specifications and tools that will allow COUNTER-compliant publishers, repositories and other organizations to record and report usage statistics at the individual article level that are credible, compatible and consistent. (Non-COUNTER-compliant organizations may use the Secondary Clearing House services described in Section 1.10 below and in Appendix D to generate PIRUS compliant usage reports from their raw usage data). COUNTER-compliant publishers may build on the existing COUNTER tools to do so, while an alternative approach is provided for non-counter compliant repositories, which is tailored to their systems and capabilities. This Code of Practice contains the following features: A list of Definitions and other terms that are relevant to recording and reporting usage of individual items A methodology for the recording and reporting of usage at the individual article level, including specifications for the metadata to be recorded, the content types, and the versions whose usage may be counted. Specifications for the PIRUS Article Reports. Data processing rules to ensure that the usage data reported are credible, consistent and compatible Specifications for the independent auditing of the PIRUS reports A description of the role of: o A Central Clearing House (CCH) in the calculation and consolidation of PIRUS usage data for articles. o Other Clearing Houses in relation to the CCH. PIRUS Code of Practice Page 1

Table of Contents 1 General Information... 4 1.1 Purpose... 4 1.2 Scope... 5 1.3 Application... 5 1.4 Strategy... 5 1.5 Governance and Relationship to COUNTER... 5 1.6 Definitions... 6 1.7 Versions... 6 1.8 Auditing and PIRUS compliance... 6 1.9 The role of the PIRUS Central Clearing House... 6 1.10 The role of other statistics clearing houses... 6 1.11 Register of PIRUS-compliant Organizations... 7 1.12 Making comments on the Code of Practice... 7 2 Definitions of terms used... 7 3 SUSHI Protocol... 7 3.1 Further information on SUSHI... 8 4 Usage reports... 8 4.1 Process for gathering individual item usage data... 8 4.2 Article Metadata... 9 4.3 Content Type... 9 4.4 Article Versions... 9 4.5 Collection and consolidation of individual article usage statistics... 10 4.6 Example PIRUS Reports for reposting usage of articles to authors & institutions... 11 4.7 Report Delivery... 12 4.8 PIRUS Central Clearing House... 13 5 Data Processing... 13 5.1 Gaming... 13 6 Auditing... 13 6.1 The Audit Process... 14 7 Compliance... 14 7.1 Timetable and procedure... 14 8 Maintenance and development of the Code of Practice... 14 9 Appendices... 15 10 References... 15 PIRUS Code of Practice Page 2

Copyright: Counter Online Metrics All rights reserved under international copyright conventions. For non-commercial purposes only this publication may be reproduced and transmitted by any means without prior permission in writing from COUNTER. All queries regarding commercial reproduction or distribution should be addressed to the Project Director, Dr Peter T Shepherd (pshepherd@projectcounter.org) PIRUS Code of Practice Page 3

1 General Information The PIRUS Code of Practice has been established as an outcome of the JISC- funded PIRUS (Publisher and Institutional Repository Usage Statistics) project 1. The primary aims and objectives of PIRUS were to assess the feasibility of and develop the technical, organizational and economic models for the recording, reporting and consolidation of usage of Journal Articles hosted by Publishers, Aggregators, Institutional Repositories and Subject Repositories. The PIRUS Code of Practice builds on the work undertaken by PIRUS, and the work of the JISC Usage Statistics Review and the Knowledge Exchange Institutional Repositories Workshop Strand on Usage Statistics. This PIRUS Code of Practice has been developed by COUNTER, which is also responsible for its ongoing management and implementation. PIRUS is consistent with the COUNTER Code of Practice. To have their usage statistics and reports designated PIRUS-compliant vendors and services must provide usage statistics that conform to this Code of Practice. A number of developments have meant that it would now be appropriate to publish a COUNTER standard for the recording, reporting and consolidation of usage statistics at the individual article level. Most important among these developments are: Growth in the number of journal articles hosted by institutional and other repositories, for which no widely accepted standards for recording and reporting usage statistics have been developed A Usage Statistics Review, sponsored by JISC ( UK Joint Information Systems Committee) under its Digital Repositories programme 2007-8, which, following a workshop in Berlin in July 2008, proposed an approach to providing item-level usage statistics for electronic documents held in digital repositories Emergence of online usage as an alternative, accepted measure of article and journal value and usage-based metrics being considered as a tool to assess the impact of journal articles. Authors and funding agencies are increasingly interested in a reliable, global overview of usage of individual articles Implementation by COUNTER of XML-based usage reports makes more granular reporting of usage a practical proposition Implementation by COUNTER of the SUSHI (Standardised Usage Statistics Harvesting Initiative) protocol facilitates the automated consolidation of large volumes of usage data from different sources. The outputs of the PIRUS project (the XML schema for the individual article usage reports, the tracker code and the associated protocols) are already being implemented by publishers and repositories (e.g. PLoS and SURF). It is important that these outputs are fully tested and, if necessary, refined, before they are too widely adopted. 1.1 Purpose The purpose of the PIRUS Code of Practice is to facilitate the recording, exchange and interpretation of online usage data at the individual article level by establishing open, international standards and 1 http://www.jisc.ac.uk/whatwedo/programmes/pals3/pirus.aspx PIRUS Code of Practice Page 4

protocols for the provision of vendor- and service-generated usage statistics that are consistent, credible and compatible. 1.2 Scope The PIRUS Code of Practice provides a framework for the recording, exchange and interpretation of online usage statistics for individual full-text journal articles. In doing so, it covers the following areas: article types to be counted; article versions to be counted; data elements to be measured; definitions of these data elements; content and format of usage reports; requirements for data processing; requirements for auditing; guidelines to avoid duplicate counting when intermediary gateways and aggregators are used. 1.3 Application PIRUS is designed for research institutions, authors, librarians, repositories and publishers and others who require reliable online usage statistics at the individual article level. PIRUS also provides publishers and repositories with the detailed specifications they need to generate data in a format useful to authors and customers. Though this Code of Practice deals with full-text articles, it is designed also be applied to other individual items of content, provided that these items meet similar data and metadata standards as full-text journal articles. 1.4 Strategy The PIRUS Code of Practice is an open standard that will evolve in response to the demands of the international scholarly publishing community (publishers, authors, librarians, researchers, research institutions or funding agencies). The Code of Practice is kept continually under review and new Releases will be published when appropriate; feedback on its scope and application are actively sought from all interested parties. See Section 8 below. 1.5 Governance and Relationship to COUNTER The PIRUS Code of Practice is owned and developed by Counter Online Metrics, a not-for-profit company registered in England. Counter Online Metrics is governed by a Board of Directors. An Executive Committee reports to the Board, and the day-to-day management of Usage Factor is the responsibility of the COUNTER Project Director, who reports to the COUNTER Executive Committee. Implementation of the PIRUS Code of Practice is not a requirement for compliance with Release 4 of the COUNTER Code of Practice for e-resources. Rather, it provides a standard that builds on COUNTER and can be implemented by those organizations wishing to record and report usage at a more granular individual item level. PIRUS Code of Practice Page 5

1.6 Definitions This Code of Practice provides definitions of data elements and other terms that are relevant to the recording and reporting of usage at the individual item level and which are consistent with COUNTER definitions. See Appendix A. 1.7 Versions This Code of Practice will be extended and upgraded as necessary on the basis of input from the communities it serves. Each new version will be made available as a numbered Release on the COUNTER website; users will be alerted to its availability. Only those organizations compliant with this Code of Practice will be considered to be providing PIRUS-compliant individual item usage statistics. 1.8 Auditing and PIRUS compliance An independent annual audit is required of each organization s reports and processes to certify that they are PIRUS- compliant. The auditing process is designed to be simple, straightforward and not to be unduly burdensome or costly to the vendor or service, while providing reassurance to customers of the reliability of the PIRUS usage data. Organizations that are currently COUNTER-compliant, e.g. publishers, will already satisfy many of the requirements of the PIRUS audit which will, in consequence, be shortened for COUNTER-compliant organizations. For organisations that are not currently COUNTER-compliant, e.g. Institutional and Subject Repositories, a subset of the PIRUS protocols are designed to allow such organisations to be become COUNTER-compliant through participation in a third-party service - the PIRUS Central Clearing House or other (national or regional) statistics aggregation service - which takes responsibility for the collection of raw usage data, the subsequent processing of those raw data into COUNTER statistics, and the supply of those COUNTER statistics back to the source organisations. In this case, only the third party service, itself, needs to be audited. The contributing organisations do not. This will result in a considerable reduction of the costs of achieving COUNTER-compliance for each organisation involved. See Section 6 below and Appendix G for more details. 1.9 The role of the PIRUS Central Clearing House The PIRUS Central Clearing House is supervised by COUNTER. The Central Clearing House has two main functions: To collect and process usage statistics at the individual article level from publishers, aggregators, repositories and other sources in order to derive consolidated PIRUS usage statistics per article To provide a central source of validated, consolidated PIRUS usage statistics for individual articles (and, potentially, other items). 1.10 The role of other statistics clearing houses Other statistics clearing houses, e.g. national or regional statistics aggregation and consolidation services, are likely to play an important role in consolidating usage data from institutional PIRUS Code of Practice Page 6

repositories and in feeding the resulting PIRUS-compliant article usage statistics to the PIRUS Central Clearing House. The role of such Secondary Clearing Houses (SCHs) will differ in detail, depending upon the context within which they operate, but their output should include the standard PIRUS Article Report 1 (Section 4.5 below), which can be fed into the Central Clearing House(CCH). In order to be an authorised provider of usage statistics to the CCH, such secondary clearing houses must be independently audited by a COUNTER-approved auditor. A PIRUS-approved model for a secondary clearing house is the IRUS-UK service a national clearing house for UK Institutional Repository statistics - described further in Appendix D. 1.11 Register of PIRUS-compliant Organizations A Register of organizations (publishers, repositories, etc.) that are compliant with the PIRUS Code of Practice is maintained by COUNTER. 1.12 Making comments on the Code of Practice The COUNTER Executive Committee welcomes comments on the PIRUS Code of Practice. See Section 7 below. 2 Definitions of terms used Appendix A lists the terms relevant to this Code of Practice and provides a definition of each term, along with examples where appropriate. In order to be designated compliant with this Code of Practice, organizations must adhere to the definitions provided in Appendix A. 3 SUSHI Protocol The SUSHI (Standardized Usage Statistics Harvesting Initiative) protocol is designed for the transmission and sharing of COUNTER usage reports. SUSHI is intended for use by publishers, aggregators and other third-party services capable of producing their own COUNTER-compliant statistics. For guidance on protocols suitable for the transmission and exchange of raw usage data, please see Appendix D. The advent of the SUSHI protocol (http://www.niso.org/workrooms/sushi/ ) has greatly facilitated the handling of large volumes of usage data and its implementation by vendors allows the automated retrieval of the COUNTER usage reports into local systems, making this process much less time consuming for the librarian or library consortium administrator. For this reason, in addition to providing the usage reports specified below (as a Microsoft Excel file, as a TSV file, or as a file that can be easily imported into Microsoft Excel pivot tables), COUNTER usage reports must also be provided in XML format in accordance with the COUNTER XML schema that is specified by the SUSHI protocol and may be found on the NISO website at: http://www.niso.org/schemas/sushi/counterelements4_0.xsd The COUNTER schema covers all the usage reports listed in Section 4 below. COUNTER reports in XML must be downloadable using the SUSHI protocol. PIRUS Code of Practice Page 7

3.1 Further information on SUSHI Further information on SUSHI is available in Appendix C of Release 4 of the COUNTER Code of Practice Comprehensive information on SUSHI is also available on the NISO/SUSHI website (http://www.niso.org/workrooms/sushi/ ). As well as full documentation on the standard itself, the SUSHI website provides: Information on Getting Started SUSHI Tools SUSHI Schemas SUSHI Reports Registry SUSHI Server Registry SUSHI Developers List SUSHI FAQs 4 Usage reports This section lists the PIRUS Usage Reports for collection, consolidation and dissemination of individual article usage data; it also specifies the content, format and delivery specifications that these reports must meet to be designated PIRUS-Compliant. A pre-condition for vendors to provide valid PIRUS usage statistics is compliance with the COUNTER Code of Practice for E-resources (www.projectcounter.org ); the usage data on which the PIRUS reports are based must be COUNTER-compliant. The information in this section is intended for use by publishers, aggregators and other third-party services capable of producing COUNTER-compliant statistics themselves. The procedures described, here, may also be followed by COUNTER-compliant repositories, but an alternative approach for repositories is described in Appendix D. 4.1 Process for gathering individual item usage data The PIRUS usage statistics must be calculated in a way that is consistent with the data processing rules specified in Section 5 of Release 4 of the COUNTER Code of Practice for E-resources. To reflect the very different technologies used by publishers/aggregators on the one hand and by repositories on the other hand, two alternative approaches to this are allowed: 1. The procedure to be followed by publishers/aggregators is described in Section 4 below. (This procedure may also be followed by COUNTER-compliant repositories.) 2. An alternative approach for repositories is described in Appendix D. This approach describes guidelines that have been developed to enable repositories to transmit raw usage data to a third party the PIRUS Central Clearing House or other statistics aggregation service - for processing into COUNTER-compliant statistics and reports. The same metadata, described in Section 4.2 below, must be recorded for each item by publishers, aggregators and repositories, irrespective of the approach used to collect the usage data. PIRUS Code of Practice Page 8

4.2 Article Metadata Publisher/aggregator organizations should collect the usage data in the format specified in Article Report 1, provided in Appendix E. The following data and metadata must be collected for each article: Either Print ISSN OR Online ISSN Article version, where available Article DOI Online Publication Date OR Date of First Successful Request Monthly count of the number of successful full-text requests - counts must remain available for at least 24 months from Online Publication Date OR date of First Successful Request The following metadata are optional, but are desirable: Journal title Publisher name Platform name Journal DOI Article title Article type 4.3 Content Type Organizations must be able to record and report usage of the following categories of journal content at the individual article level: research articles ( full articles and short communications) review articles In addition the usage reports on the following individual items are acceptable, provided they meet the data and metadata requirements listed in 4.2 above: editorials book reviews theses 4.4 Article Versions Only usage of the following 5 Article Versions (of the 7 versions defined by the ALPSP/NISO JAV Technical Working Group (http://www.niso.org/publications/rp/rp-8-2008.pdf)) may be counted: Accepted Manuscript (AM) Proof (P) Version of Record (VoR) Corrected Version of Record (CVoR) Enhanced Version of Record (EVoR) Usage of the following 2 Article Versions must not be counted: Author s Original (AO) Submitted Manuscript Under Review (SMUR) PIRUS Code of Practice Page 9

4.5 Collection and consolidation of individual article usage statistics Publishers, aggregators and COUNTER-compliant repositories must collect individual article usage data in the format specified in Article Report 1, below. Those publishers wishing to report consolidated usage statistics for their individual full-text articles and other items, based on usage data from a number of sources in addition to the publisher s own platform (e.g. aggregators, subject repositories, or institutional repositories) may do this consolidation themselves, or may use the Central Clearing House to fulfil this function. Publishers wishing to use the Central Clearing House for this purpose must supply it with usage data in the format specified in Article Report 1 below of this Code of Practice. Only COUNTER-compliant publishers shall have the right to submit usage data to the Central Clearing House. PIRUS Article Report 1: Publisher specification for data collection by article (See Excel Spreadsheet in Appendix E) Note: 1. Article title data is highly recommended, but opional 2. Usage data should include: a. Include: successful full-text requests (HTML plus PDF) b. Include: Accepted Manuscript, Proof, Version of Record versions c. Exclude: Author s Original Manuscript and Submited Manuascript Under Review versions d. Exclude: any internal use by publisher and host, downloads from LOCKSS caches, and usage driven by robots PIRUS Code of Practice Page 10

4.6 Example PIRUS Reports for reposting usage of articles to authors & institutions This section lists the PIRUS Reports that publishers and repositories must provide to their authors and institutions in order to be designated PIRUS-compliant. Article Reports 2 and 3 provide standard formats for the reporting of individual article usage statistics to authors and their institutions. Specified are the data they must include, as well as the format they must adopt. While these examples are in Excel format, (See Section 4.4 below for other report delivery options), primarily for visualisation purposes, all PIRUS usage reports must be available in XML, irrespective of other formats provided. Reports must comply exactly with the formats specified in order to be PIRUS compliant. PIRUS Article Report 2: Number of Successful Full-Text Article Requests by Author, Month and DOI, consolidated from different sources (See Excel Spreadsheet in Appendix E) Note: 1. The ORCID Identifier should be used to identify the author Article title data is highly recommended, but opional 2. Usage data should include: a. Include: successful full-text requests (HTML plus PDF) b. Include: Accepted Manuscript, Proof, Version of Record versions c. Exclude: Author s Original Manuscript and Submited Manuascript Under Review versions d. Exclude: any internal use by publisher and host, downloads from LOCKSS caches, and usage driven by robots PIRUS Article Report 3: Summary of All Successful Individual Article Requests for an author, by month (See Excel Spreadsheet in Appendix E) PIRUS Code of Practice Page 11

Note: 1. The ORCID Identifier should be used to identify the author 2. Article title data is highly recommended, but optional 3. Usage data should include: a. Include: successful full-text requests (HTML plus PDF) b. Include: Accepted Manuscript, Proof, Version of Record versions c. Exclude: Author s Original Manuscript and Submited Manuascript Under Review versions d. Exclude: any internal use by publisher and host, downloads from LOCKSS caches, and usage driven by robots 4.7 Report Delivery Unless specified otherwise, all PIRUS reports must conform to the following standards: Reports must be provided in the following formats: o Microsoft Excel file (see Section 4.1 above), or as a TSV file, or as a file that can be easily imported into Microsoft Excel pivot tables. o As XML formatted in accordance with the PIRUS schema (available at: http://www.niso.org/schemas/sushi/counterelements4_0.xsd ). More information on XML formatting is available in Appendix H. Each report should reside in a separate file or page to avoid files of unwieldy size Reports should be made available on a password-controlled website (accompanied by an email alert when data is updated). Reports must be readily available PIRUS Code of Practice Page 12

4.8 PIRUS Central Clearing House Publishers may consolidate the PIRUS individual article usage data from various sources themselves, or they may use the Central Clearing House for this purpose. The main features of the Central Clearing House are: A home page that provides summary information on: o the number of articles and journals indexed o overall totals of successful full-text article requests recorded from publishers, repositories and other organizations A Search facility that makes it possible to find individual articles or groups of articles by: o DOI o Title/Author A number of reports can be generated: o Article Report 1j (AR1j) this is a variant of the Article Report 1 (AR1), with usage events restricted to one journal at a time to reduce the report sizes. Its main purpose was to allow easy cross-checking that the data exposed from the PIRUS database matches the original data supplied by publishers o Article Report 2 (AR2) this report is intended for article authors showing usage consolidated from publishers and repositories for their articles Each of the reports may be viewed in a web page in the portal or downloaded for use locally as MS- Excel/TSV files (See Appendix E for examples of the various reports). These reports must also be available via the SUSHI protocol. 5 Data Processing Usage data collected by organizations must be in the formats specified in Article Report 1 and be consistent with the COUNTER Code of Practice. The data processing rules to be followed will be found in Section 5 of Release 4 of the COUNTER Code of Practice for e-resources. 5.1 Gaming As attempts to game individual article usage statistics are not unlikely, the following measures are necessary to counteract the potential inflationary effects of such gaming. COUNTER protocols against machine-driven gaming A programme of statistical forensics to identify suspicious behaviour, both human and machine-driven in origin 6 Auditing An important feature of the PIRUS Code of Practice is that compliant vendors must be independently audited on a regular basis in order to maintain their PIRUS listing. This audit is supplementary to the existing COUNTER audit and is designed to validate those aspects of the PIRUS data processing and presentation not already covered in the COUNTER audit. These supplementary auditing standards and procedures have been published in Appendix G of this Code of Practice. As with COUNTER, these audits will be conducted online using the detailed test scripts included in the auditing standards and procedures. PIRUS Code of Practice Page 13

The independent audit is required within 6 months of publishers first achieving a listing on the Register of PIRUS-compliant vendors (see Section 7.1 below), and annually thereafter. COUNTER will recognize a PIRUS audit carried out by any CPA (Certified Public Accountant) (USA), by any CA (Chartered Accountant) (UK), or by their equivalent in other countries. Alternatively, the audit may be done by another, COUNTER-approved auditor, such as ABC, which is not a CA or a CPA. Information on COUNTER-approved auditors is available from COUNTER. 6.1 The Audit Process 1. PIRUS-compliant publishers and services will be notified in writing that an audit is required at least 3 months before the audit is due. 2. Publishers and services should respond within 1 month of receiving the reminder by informing COUNTER of their planned timetable for the audit and the name of the organization that will carry out the audit. Any queries about the audit process may be raised at this time. 3. Irrespective of the auditor selected, the audit must adhere to the requirements and use the tests specified in Appendix G of this Code of Practice. 4. Upon completion of the audit the auditor is required to send a signed copy of the audit report to the COUNTER office (pshepherd@projectcounter.org). 5. Categories of audit result are as follows: a. A Pass, in which case no further action is required by the publisher or service as a result of the audit. In some cases the auditor may add Observations to the audit report, which is designed to help the publisher/service improve its PIRUS reporting, but which are outside the scope of the audit itself. b. A Qualified Pass, in which the auditor deems the vendor or service to have passed the audit, but where the auditor raises a Minor Issue requiring further action to maintain a valid listing in the PIRUS Register. A Minor Issue does not affect the reported figures, but is one which should be resolved within 3 months of the audit to COUNTER s satisfaction. An example of a Minor Issue is where a report format does not conform to the PIRUS specifications. c. A Fail, where the auditor has identified an issue that must be resolved immediately for the publisher or service to maintain a valid listing in the PIRUS Register. 7 Compliance 7.1 Timetable and procedure Release 1 of the PIRUS Code of Practice, published in final form in XXXX, will become the only valid version of this Code of Practice from YYYY. 8 Maintenance and development of the Code of Practice COUNTER has overall responsibility for the development and maintenance of this Code of Practice. Each new Release will be made openly available in draft form on the COUNTER website for comment PIRUS Code of Practice Page 14

before it is finalised. Comments may be sent to the COUNTER Project Director, Dr Peter T. Shepherd at pshepherd@projectcounter.org. When providing your comments you are requested to adhere to the following guidelines: Please be as specific as possible, making sure to note the relevant section and subsection of the Code of Practice. Where you are proposing an addition to the Code of Practice, please indicate the preferred section within the current version. 9 Appendices Appendix A: Glossary of Terms relevant to PIRUS Appendix B: Vendor Declaration of PIRUS Compliance Appendix C: Specification for Gathering of PIRUS data Appendix D: Protocols for Repositories Appendix E: Excel usage reports examples Appendix F: Guidelines for implementation Appendix G: Auditing requirements and tests Appendix H: XML overview, with links to the up to date schemas 10 References 1. Needham, P., and Shepherd, P., Publisher and Institutional Repository Usage Statistics: the PIRUS2 Project: http://www.projectcounter.org/news/pirus2_oct2011.pdf PIRUS Code of Practice Page 15