IMLS National Leadership Grant LG Copyright Review Management System

Transcription

1 IMLS National Leadership Grant LG Copyright Review Management System University of Michigan Library Interim Performance Report: Narrative - (12/20/2012) Overview Some useful statistics: CRMS- US CRMS- World Volumes remaining to review ~ 74,800 ~ 159,200 Actual determinations made ~ 40,600 ~ 20,600 Percentage of determinations: PD ~ 55% ~ 57% Percentage of determinations: UND ~ 28% ~ 27% The Copyright Review Management System World (CRMS- World) project is building a system that enables the University of Michigan Library and its partner institutions to investigate the copyright status of books published in Australia, Canada, and the United Kingdom between 1872 and 1941 and to continue reviewing books published in the United States between Books identified as being in the public domain (no longer subject to copyright) will be opened in HathiTrust, meaning made readable by the general public. Additionally, we are engaged in related efforts to advance our knowledge of the copyright status of books in library collections through national and international dialogue that will improve legal access to and uses of cultural heritage and public domain materials. As part of this dialogue, we are in preliminary discussions with the Humboldt- Universität zu Berlin, where interested faculty are beginning to develop a rights identification project. We are also working with state document librarians in California, Michigan, and Utah in order to make state government documents available from the HathiTrust Digital Library. A key part of this project is the inclusion of a broad coalition of libraries. Their participation is fundamental to the success of CRMS and demonstrates the importance of copyright determination work for libraries. Our partners include Baylor University Library, California Digital Library, Columbia University Libraries, Dartmouth College Library, Duke University Libraries, Indiana University Libraries, the Sheridan Libraries of Johns Hopkins University, Northwestern University Library, Penn State University Libraries, Princeton University Library, UC Irvine Libraries, UCLA Library, UCSF Library, University Library of the University of Illinois at Urbana- Champaign, University of Maryland Libraries, University of Minnesota Libraries, and University of Wisconsin- Madison Libraries. 1

2 Project Activities: Summary of Activities, December November 2012 Define requirements, design, and develop the CRMS- World database, workflow, management system, and web interface. Test iteratively and revise system as needed. Over the first year of the grant (and during the pre- grant period) we defined, designed, and developed the theoretical and technical back ends of the project. This included the legal methodology, the CRMS- World web interface, and the systems required to train, manage, and support the reviewers. These systems are now in place and are being tested and updated iteratively through the grant period. Figure 1: The CRMS- World Interface To increase efficiency without sacrificing effectiveness and accuracy, we adjusted the selection criteria for candidate volumes as we received new information pertaining to the parameters used in the review process. For example, we removed translations from the candidate queue due to complicated rights questions, and we added second and later editions when we saw no demonstrated reason to restrict our reviews to first edition works. (See Project Analysis, Lessons Learned, below, for more on editions.) 2

3 Develop a training plan and materials. Train reviewers from partner institutions to join in collaborative copyright review work. We developed and released a CRMS- World Training Module in Spring This module acquaints new reviewers with the project, interprets the review interface, and explains the decision tree used in the review process. The module was designed to support both face- to- face and distance training. At first, we assumed new reviewers would need to train themselves because project team members would not be able to visit every institution that might participate. In practice, however, those who have joined the program in the past seven months have been affiliated with the partner institutions that were represented at the training meeting in May We have, in effect, trained the trainers, and this means the training module has been a supplement to on- site training more than a stand- alone resource. Further supplementary training support is available through a community listserv, a regularly updated wiki, and frequent communication between the program manager and new reviewers. We are currently developing short training videos to demonstrate different parts of the review process, and we are refining all of our training materials to make them easier to use for remote training. We intend to make all training materials available on the project website. Perform copyright review work using CRMS- World and CRMS- US. In May 2012, we hosted a three- day meeting at the University of Michigan Library for 22 people from our partner libraries. The meeting provided them with face- to- face training to become CRMS- World reviewers. In the months that followed, the reviewers scaled up to full production, and they now contribute consistent percentages of time each week to the project. Concurrently, the CRMS- US team continues to work at full production and submit reviews in the now well- established copyright determination process. Additionally, we have implemented "rights inheritance" programming for volumes that are duplicates of those that have undergone review. In other words, if there are multiple copies of the same book in CRMS- US, the determination for one book will flow to the others. In CRMS- World, "rights inheritance" ensures that once we have identified a death date or similar triggering date for one book, we can apply the same information to all books by that same author without additional research. (However, we do confirm that all of the books are subject to the same assumptions. For example, if the author contributed to a book with multiple authors, it may enter the public domain at a different time than his other works.) Continuously monitor progress and track areas to identify opportunities for ongoing improvement. Resolve issues as they arise. Feedback from the initial round of reviews helped to refine the CRMS process. For example, in response to suggestions from reviewers, we improved the CRMS- World interface to include more effective search tools, additional rights codes, and more sophisticated ways of gathering data. Some adjustments were simple. For example, if the reviewers found a given resource more useful than the 3

4 others, we moved it up in our list of search tools so it would be more visible and easier to access. Reviewers also discovered technical bugs which could then be fixed by our software developer and the project management staff. We created two data visualization dashboards that show the progress of the project in several respects. The dashboards' tables and charts are updated each day, and they have been helpful for tracking progress and gathering statistics. Figure 2: The CRMS- World Dashboard Monitor productivity and improve the accuracy of copyright determinations. Introduce a Decision Tree to establish an order of operations for the criteria used to make reliable rights determinations. We monitor reviewer accuracy and seek to ensure an effective and responsible review process as part of our supervision of the project. We developed and distributed a CRMS- World Decision Tree that establishes an order of operations for the review process and emphasizes for reviewers the importance of following an established and consistent method. The decision tree's method is also efficient. For example, the first step is a comparison of the scan with its corresponding catalog metadata, ensuring that a reviewer will not waste time and effort reviewing an erroneous or mismatched record. 4

5 Figure 3: The CRMS- World Decision Tree Each volume in CRMS is reviewed independently by two different reviewers. If the two reviews conflict, the work is referred to an expert reviewer for resolution. The role of the expert reviewer is to inspect the two reviews and determine how to handle the specific situation. Both reviews may agree on a volume's rights status but disagree on the reasons for that status. This is often what causes a conflict in the system. Resolving review conflicts is a key part of the quality control process. 'Expert reviewers' also give useful feedback to reviewers when examining their historical work. 5

6 Add functionality to the system to accommodate the training and integration of reviewers at outside institutions. As mentioned above, we have developed mechanisms to facilitate off- site training, such as online training materials, a wiki with responses to common reviewer questions, and an active listserv where reviewers can ask questions and share successful strategies with each other. We have also developed and implemented systematic procedures for enabling secure access to the review interface, the wiki, and the listserv in order to ensure the quick and efficient training and integration of new staff. Publicize efforts website; regular updates to stakeholders and interested parties; articles/interviews; presentations. We have several venues for publicizing our efforts and communicating with interested parties. We provide a monthly summary of statistics that appears in the HathiTrust Update newsletter and is posted on the HathiTrust News and Publications page located at < The project website contains links to documentation and presentations and can be found at < national- leadership- grant- crms- world>. We are submitting proposals to organizations such as CNI and ALA to offer presentations at their conferences about the goals and accomplishments the CRMS projects. Past presentations about CRMS can be found on the project website at < national- leadership- grant- crms- world/presentations- and- resources>. Our work has captured the attention of librarians. Bernadette Bartlett, Michigan Documents Librarian, at the Library of Michigan and Kris Kasianovitz, Government Information Librarian, at Stanford University, will be presenting at the 2012 Best Practices Exchange in Annapolis Maryland. Both talks discuss aspects of CRMS efforts to address the copyright status of U.S. state government documents in HathiTrust. Develop methods for sharing findings on copyright status. We have had preliminary discussions with the Harry Ransom Center, the Stanford Copyright Review Database, and the United States Copyright Office about the possibility of sharing copyright information and other forms of metadata. We are contributing to the US Copyright Office s discussion about improving access to copyright registration documentation in the hope that this will ease the cost and complexity of copyright determinations in the future. Reach out to state documents librarians in California, Michigan, and Utah to expand the project scope to state government documents. Over the past months, we developed contacts with librarians at the State Library of Michigan and are working to make available many State of Michigan government documents. (While the work of federal employees is in the public domain, work by state employees is subject to state copyrights.) We are in the process of opening documents where possible. As noted above this effort will be presented at the 2012 Best Practices Exchange conference in Annapolis, Maryland. We hope that by 6

7 sharing our successes in this manner we will encourage other state government partners to join the project and make more state government works available in HathiTrust over time. Examine the utility of the Durationator, a copyright duration calculator developed and maintained by Professor Elizabeth Townsend Gard of Tulane University. In order to have an independent analysis of our review process, we provided three sample sets of 50, 100, and 200 volumes reviewed in the CRMS- World process to Professor Townsend Gard of Tulane University Law School. Her Durationator team ( conducted an independent copyright status analysis on these records and then compared their results to the determinations made by CRMS- World. The findings of the Tulane team matched 100% of the public domain determinations by CRMS- World. Project Audience CRMS has two target audiences: (1) users of public domain collections including scholars, students, educators, readers, and lifelong learners worldwide, and (2) people or institutions that may want to adopt our approach in order to undertake rights determinations of their own. For example, the California Digital Library volunteered to join us as a partner, and we have had positive responses from state government documents librarians. Both groups now actively participate in the CRMS- World project. As we move forward with the next phase of the project, we hope to gain additional partners who can contribute to the CRMS- World determination process. Students are a key part of our audience, and we have worked with graduate students at the University of Michigan and Stanford University. We have helped students make use of HathiTrust resources for their research and provided them with research opportunities that have contributed to the CRMS- World project. Project Analysis We are fully up to date on all tasks scheduled in the grant timeline. We are pleased to share our progress. a. Targets As of December 7, 2012, the CRMS- US reviewing team has conducted over 350,000 reviews, resulting in copyright determinations for over 40,000 volumes in HathiTrust since Meanwhile, in the seven months since the CRMS- World training meeting, the CRMS- World reviewers have conducted approximately 43,000 reviews resulting in determinations for 7

8 over 21,000 volumes. CRMS- World now has over twenty trained reviewers from seventeen partner institutions, and the project team has opened a dialogue with other potential partners. Selected partner institutions have trained additional reviewers, with oversight and training materials provided by the CRMS- World project team. i. Achievements We fulfilled all of the major milestones set during the first year of the grant. Our reviewers met in Ann Arbor for two days of training that prepared them for conducting reviews and allowed the project team to create materials to support distance training. We also formed our Advisory Board. In a two- day meeting in September, the advisors approved the review method, addressed major copyright issues relevant to CRMS- World, and recommended important refinements to the process regarding editions, terminal dates, and author death dates. We have made real progress toward identifying and recruiting new international partners. We are developing a process for determining Spanish works, and we are talking with Professor Michael Seadle, Dean of the School of Library and Information Science at Humboldt- Universität zu Berlin, regarding books in German. We have recruited additional domestic partners, with the California Digital Library joining the project in June of As noted above, we have also been working with the Durationator Project, under the direction of Professor Elizabeth Townsend Gard of Tulane University, to verify the accuracy of our determination process. ii. Lessons Learned from Unanticipated Events There are distinct differences between the CRMS- World and CRMS- US research methods. Some of our preliminary assumptions, based on our experience with CRMS- US, required modification when the CRMS- World team began actively reviewing. For example, first edition status is vital for a CRMS- US review, and we assumed the same would be true for CRMS- World reviews. However, an analysis of our review data and discussion with our Advisory Group made it clear that first edition status was not necessary for a valid CRMS- World inquiry. The process was modified accordingly. The 'expert reviewer' step in the CRMS- World process has been a temporary "bottleneck" during the first year. There were two reasons for this: 1) a precautionary decision to oversee all of the judgments being made by a new group of reviewers, and 2) having fewer expert reviewers by comparison with CRMS- US. Fortunately, both of these concerns have already been addressed. The reviewers quickly proved themselves to be a reliable team, 8

9 and we have added another expert reviewer to CRMS- World, with plans to begin training a third expert review early next year. Preliminary data suggest we may be able to make categorical assumptions about the pool of candidate volumes awaiting CRMS review. For example, about 20% of the candidate volumes, those published between 1872 and 1881, have never resulted in an in copyright determination. For those volumes published between 1882 and 1890, fewer than twenty have been found to be in copyright. In response to this analysis, we are considering the possibility of changing the terminal date used for presumptively treating non- US works as in the public domain (currently 1872, rolling forward each year). The CRMS- World review method provides reviewers with several sources for confirming author death dates. The "Library of Congress Authorities" has been the most reliable of these sources. The others have also been useful, but they vary in the amount of detail they provide about authors (for example, most of them do not provide titles associated with a given author, a feature which would aid identification significantly). This lack of useful detail among our sources introduced a persistent degree of uncertainty in our method and generates conflict between reviewers. The CRMS- World team continues to look for better alternatives, and we are exploring the possibility of reaching out to some of the teams supporting these sources in order to help them improve the quality of the information they provide. What s Next? We have several initiatives planned for 2013, in addition to our continued effort in making copyright determinations: Rights Research: The CRMS project has established reliable methods for determining the copyright status of works in the HathiTrust. To complement these methods, a "Rights Research Project" would conduct research to identify the actual rightsholders of these works. Rightsholders (including authors, publishers, and other parties) could then be contacted and asked if they would be willing to open their works in HathiTrust. Partnerships: We will continue to cultivate additional partners. We are particularly interested in establishing partners outside the United States, which would allow us to address volumes in non- English languages that were published in foreign countries. State government documents: We intend to expand our collaboration with state government librarians to make government documents more available in HathiTrust. 9

10 Outreach and presence: Members of the CRMS- World team will continue to attend conferences and publish articles to raise the visibility of the CRMS projects and their mission. Planned changes in the UM Library's infrastructure should also allow us to make important additions to our online "public face," including dashboards that will display the achievements of the project in charts and graphs. Audits: We are developing a new audit regime that will allow us to confirm the reliability of CRMS- World reviews and the determination process. This effort will rely on established methods applied in previous HathiTrust- related audits as well as best practices used in records management. 10