LIS 60632: Technologies for Digital Preservation and Web Archiving. Fall 2010

Transcription

1 LIS 60632: Technologies for Digital Preservation and Web Archiving Fall 2010 Instructor: Dr. Karen F. Gracy Course Website: In-Person Meetings: Saturday, October 30, 2010, 9:00 AM 5:30 PM, 315 Library Saturday, November 20, 2010, 9:00 AM 5:30 PM, 315 Library Saturday, December 4, 2010, 9:00 AM 5:30 PM, 315 Library Contact with the Instructor: Messages and Questions Phone: ; (please use the Vista function for class-related questions). The instructor strives to respond to e- mail and phone calls within 48 hours. Office Hours In person: By appointment. Chat: Mondays and Wednesdays, 11:00 a.m.-12:00 p.m. and 1:00-2:30 p.m. via Skype Chat (download Skype at instructor s Skype Name is kgracy). Video call: Instructor will also be available for virtual office hours by appointment using Skype for video-enhanced telephone calls (download Skype at instructor s Skype Name is kgracy). Registrar s Deadlines: Last Day to Drop: October 31, 2010 Last Day to Withdraw: November 27, 2010 Course Description: Prerequisite: LIS or LIS In this course students will gain proficiency in essential technologies for building and maintaining robust, trusted digital preservation repositories. Emphasis is on providing orientation to technologies sufficient to allow students to work with network 10/29/2010

2 administrators, programmers, and other personnel involved in providing technical support to develop digital repositories for preservation and archiving functions. Students will also evaluate technologies for suitability in digital library and archiving environments. Technologies to be addressed in the course include: Metadata harvesting, extraction, and editing tools; File format identification and validation tools; Tools for authentication of files; Migration and emulation software; Software for web archiving and curation. Course Objectives: By the time students complete this course, they should be able to accomplish the following primary objectives: Identify functional needs for a preservation-ready digital repository; Assess the suitability of toolsets and other software for implementation in preservation activities; Differentiate among different types of metadata schemas and relate them to functional needs for a preservation-ready digital repository; Analyze institutional needs for preservation repository architecture and make recommendations for metadata standards, repositories, and other technologies suitable for achieving preservation goals; Differentiate among different approaches to web archiving and understand the basic models for web capture and search of archived materials. Required Resources: All required readings will be made available on the Vista website (as PDFs or links to online resources). No textbook is required for this course. Course Requirements: This course will require students to complete several types of work associated with weekly learning goals: 1. Weekly assignments (reviews/reports/case studies; 4 total, 3 worth 10% and 1 worth 15% of the final grade); 2. In-class exercises (3 total, each worth 5% of the final grade); 3. Blogging posts (5 weeks total, each week s participation worth 5% of the final grade); 4. Web archiving final project (worth 15% of the final grade). 2

3 Weekly Assignments Week 2 Assignment #1: METS Toolkit Review Week 4 Assignment #2: Preservation Planning Case Study (Knowledge Management Working Group Webinar Archive) Week 5 Assignment #3: Evaluation of Emulation Software Week 7 Assignment #4: Web Harvester Review These papers will be five to seven pages in length. More detailed instructions for completing these assignments may be found in this syllabus and on the Vista website. Grading rubrics are also available on Vista to inform you of how your work will be evaluated. Failure to complete any of the assignments will result in a failing grade in the course. Please carefully proofread your assignments for accuracy in spelling and grammatical construction such errors do play a part in your grade, particularly if they affect the instructor s ability to understand the content or import of your writing. In-Class Exercises Week 1 Exercise #1: Transfer/ingest Week 4 Exercise #2: Preservation planning Week 6 Exercise #3: Web capture For those weeks that we will have a F2F session, we will spend afternoons working on a hands-on exercise. You must be present to get credit for these exercises. Final Project The final project for the course will be a web archiving assignment, due the final week of the course. More details about this assignment can be found on Vista. Participation (Weekly Blogs and Face-to-Face Sessions) For all weeks that there is no face-to-face meeting (Weeks 2, 3, 5, 7, and 8), you must contribute postings to your blog and respond to classmates blogs (topics will be posted on Vista). Please consult the rubric on Vista to see how blog participation will be evaluated. You must complete blog postings for each online week to receive full credit toward your participation grade. For F2F sessions, class participation will be measured by the quality and substance of your contributions, including: a. evidence of having read assigned readings; b. asking meaningful and relevant questions; c. volunteering answers and ideas; and, d. active involvement in group activities and exercises. 3

4 Weighting of Assignments and Assessments The following table summarizes all assignments and assessments, and how they will be weighted to determine your final grade. Please take in consideration the weighting of each component. Points awarded for a component will be normalized to equal what your weighted grade would be on a 100-point scale. In some instances, there is no need to normalize (i.e., if the top grade for an assignment is 5 points and it s worth 5% of your total grade, then each point is worth 1% of your grade!). Summary of Assignments & Assessments (with weighting percentages) Assessment Criteria Course Participation Hands-On Exercises (Completed In- Class) Assignments Component Percentage Due Date Online blog postings (5 topics, each worth 5% of total grade, in Weeks 2, 3, 5, 7, and 8) Week 1: Transfer/Ingest Exercise Week 4: Migration and Emulation Exercise Week 6: Web Capture Exercise #1: METS Toolkit Review #2: Preservation Planning Case Study (Knowledge Management Working Group Webinar Archive) #3: Evaluation of Emulation Software #4: Web Harvester 25% Initial posting due by Friday at midnight of that week; comments on other classmates postings due by Sunday at midnight. 5% October 30, % November 20, % December 4, % November 8, % November 29, % December 6, % December 13, 2010 Review Final Project Web Archiving Project 15% December 17, 2010 Total 100% 4

5 Final Grades Final grades for this course will be assigned as follows: A = points A- = points B+ = points B = points B- = points C+ = points C = points F = 73.9 points or below Policies: The instructor expects students to adhere to the following policies: Required class attendance. According to University policy, regular attendance in class is expected of all students. Legitimate reasons for absence include illness, death in the immediate family, religious observance, academic field trips, and participation in approved university-related events (please see the full policy at o Special notes for students taking hybrid courses (hybrid = online components + face-to-face components): If you do not attend the first in-person meeting, you will receive an NF grade for the course. NF = Failed (never attended). For weeks where there are no F2F meetings, you must login to the Vista website at least once a week and show evidence of activity (such as posting a comment on a discussion board, uploading an assignment, or taking a quiz). If you do not login to the Vista website for two weeks in a row (a period of 14 days), you will receive an SF grade for the course. SF = Failed (stopped attending). Full participation in class discussion and exercises. Be prepared to contribute to the learning process by completing readings and any associated exercises before making contributions to discussions; Timely completion of all assignments. Late assignments will be marked down 1/2 grade for each day they are turned in past the due date, unless prior arrangements have been made with the instructor; assignments more than two days late will not be accepted. Students are expected to conduct themselves in a business-like and courteous manner at all times. Please consult the policies on group work and discussion boards (called online etiquette, found in the Before You Begin folder on the Vista course website), which detail appropriate conduct in these venues. 5

6 Special Notes (1) Technological Competencies Students must be familiar with basic computer operations (e.g., copying and printing files, moving among directories and subdirectories), logging on to a network, using a modem and/or an Internet Service Provider, and using word processing software. (2) Students with Disabilities In accordance with University policy, if a student has a documented disability and requires accommodations to obtain equal access for a course or workshop, he or she is responsible for notifying the instructor at the beginning of the course or workshop in which an accommodation is required. Please note that it is necessary for the student to first verify his/her eligibility for requested accommodations through the Office of Student Accessibility Services (SAS) located on the ground floor of the DeWeese Health Center at the Kent Campus ( ). To do this, he or she must schedule an appointment with an SAS staff member and provide the SAS office with appropriate documentation of his or her disability. Upon verification, the SAS staff member will present the student with "accommodation letters" to give to his or her instructors. For questions, please contact Mary Reeves, Director of University Health Services and Student Accessibility Services: (3) Policy on Incomplete Grades A complete statement on INCOMPLETE grades is available upon request. In summary, it states that the student must be earning a "C" (2.0) grade or better and be unable to complete the work before the end of classes due to extenuating circumstances. The request for an INcomplete grade is not to be used as an excuse for making up for poor performance. The student must initiate the request, provide appropriate documentation, and make arrangements to make up the incomplete work. For undergraduate students, Incompletes must be made up by the end of the following semester, and for graduate students, Incompletes must be made up within one calendar year, or an extension granted, otherwise a default grade of "U" will be assigned. However, while the university policy is set for a maximum for one semester for undergraduate students and one calendar year for graduate students, the instructor has the right to set earlier deadlines, based on circumstances and other considerations. (4) Cheating and Plagiarism Both cheating and plagiarism are prohibited. One area that many students may not realize as cheating is the following. "Using a substantial portion of a piece of work previously submitted for another course or program to meet the requirements of the present course or program without notifying the instructor to whom the work is presented." See the University Policy Register for additional details on cheating or plagiarism: 6

7 Weekly Schedule of Topics Covered Week 1 (October 25-31, 2010) Please consult Vista for preparatory readings and activities to be completed prior to our first class session. F2F Meeting: Saturday, October 30, 2010, 9:00 AM 5:30 PM, 310A Library Overview of Course (9:00-9:45 am) Review of Digital Preservation Basics (9:45-10:30 am) Open Archival Information System (OAIS) Preservation metadata requirements o PREMIS and METS Workflows and micro-services Readings: Reference Model for an Open Archival Information System, Priscilla Caplan, Understanding PREMIS, Library of Congress, Steve Hitchcock, et al., Digital Preservation Service Provider Models for Institutional Repositories, D-Lib Magazine 13.5/6 (May/June 2007). METS: An Overview and Tutorial, Rainer Schmidt, et al., A Framework for Distributed Preservation Workflows, International Journal of Digital Curation, Issue 1, Volume 5, Break (10:30-10:45 am) Transfer and Ingest Accepting SIPs (10:45 am-12:00 pm) File encapsulation for transfer Quarantine File format identification File format validation 7

8 Metadata extraction File normalization Readings: Adrian Brown, Selecting File Formats for Long-Term Preservation, Malcolm Todd, File Formats for Preservation, DPC Technology Watch Series Report 09-02, October Resources: File Format Identification and Validation Tools, DSpace METS Document Profile for Submission Information Packages (SIP), Lunch (12:00-1:30 pm) SIP Laboratory (1:30-5:30 pm) Laboratory will include introduction to several toolkits that facilitate file transfer, format identification and validation, and other digital preservation activities, including: 1. Library of Congress Transfer Tools. (Bagger, which uses BagIt specification) 2. Digital Preservation Software Platform (National Library of Australia). Includes: a. Xena b. Checksum Checker c. Digital Preservation Recorder 3. FITS (File Information Tool Set): identifies, validates, and extracts technical metadata for various file formats. Includes: a. DROID b. JHOVE2 c. NLNZ Metadata Extraction Tool 4. Archivematica: comprehensive digital preservation system that uses an integrated suite of free and open-source tools that allows users to process digital objects from ingest to access in compliance with the ISO-OAIS functional model. 8

9 Week 2 (November 1-7, 2010) Metadata Requirements Creating AIPs METS How well it plays with others (Review of various metadata schemas) Readings Angela Dappert and Markus Enders, Using METS, PREMIS and MODS for Archiving E-Journals, D-Lib Magazine 14.9/10 (2008) Rebecca Guenther, Battle of the Buzzwords: Flexibility vs. Interoperability When Implementing PREMIS in METS, D-Lib Magazine 14.7/8 (2008): Sally Vermaaten, A Checklist and a Case for Documenting PREMIS-METS Decisions in a METS Profile, D-Lib Magazine 16.9/10 (2010): Resources Content schemas (e.g., MARC/MODS/Dublin Core/VRA, etc. as Descriptive Metadata in METS) Structural/Technical schemas (e.g., METS, MIX, TextMD) o <METS> METADATA ENCODING AND TRANSMISSION STANDARD: PRIMER AND REFERENCE MANUAL, version 1.6 (Sept. 2007), o MIX (NISO Metadata for Images in XML Schema), o TextMD (Technical Metadata for Text), Preservation schemas: PREMIS as Administrative Metadata in METS) o PREMIS in METS Toolbox, Other format-specific content and technical schemas, such as: o AES Audio Object Schema (in development) See Jane Johnson Otto, A Sound Strategy for Preservation: Adapting Audio Engineering Society Technical Metadata for Use in Multimedia Repositories, Cataloging & Classification Quarterly 48:5 (2010): o PBCore, 9

10 o Material Exchange Format (MXF), Progress Report, 1.pdf Technical Overview, 2.pdf o LC-AV for Audio and Video METS extension schemas (for describing characteristics of analog originals prior to digitization) o SMPTE RP210 Metadata Dictionary (video), Online Activities Weekly Blog o Topic: Look at a metadata schema other than Dublin Core, METS, or PREMIS in-depth. Answer the following questions in your blog post: Who developed the schema? Who is implementing it and in what ways? How well does it play with other schemas? What is its value for preservation activities? What are its limitations for implementation? Assignment #1: Metadata Toolkit Review, due November 8, o Topic: Choose a metadata toolkit and write a critical review at least five pages in length that identifies its capabilities and limitations and makes recommendations about its applications in digital repository environments: PREMIS in METS Toolbox, Archivists Toolkit, OpenWMS, Sobek CM METS Editor, OpenMIC, MediaInfo, PBCore Instantiationizer, BWF Metaedit (permits embedding, validating, and exporting of metadata in Broadcast WAVE Format (BWF) files), 10

11 Week 3 (November 8-14, 2010) Storage and Access Technologies Storage Managing AIPs Auditing Validation checks Authenticity checks Readings Ross Harvey, Digital Curation (New York: Neal-Schuman, 2010). Chapter 14: Storing Data. Steven Abrams, et al., What? So What : The Next-Generation JHOVE2 Architecture for Format-Aware Characterization, International Journal of Digital Curation, Issue 3, Volume 4, 2009, Michael Factor, et al., Authenticity and Provenance in Long Term Digital Preservation: Modeling and Implementation in Preservation Aware Storage, TaPP 09, First Workshop on the Theory and Practice of Provenance, San Francisco, Audrey Novak, Fixity Checks: Checksums, Message Digests, and Digital Signatures, 2006, Maik Schott et al., Integrity and Authenticity for Digital Long-Term Preservation in irods Grid Infrastructure, Virtual Goods conference proceedings, Polzan, Poland, 2008, Petros Maniatis et al., Preserving Peer Replicas By Rate-Limited Sampled Voting, Resources JHOVE2, Jacksum, Checksum Checker, part of the Digital Preservation Software Platform, Access Supporting search and retrieval o Generating reports of search results o Generating DIPs Sharing AIPs among preservation repositories, using Repository exchange Packages (RXPs) 11

12 Readings Ross Harvey, Digital Curation (New York: Neal-Schuman, 2010). Chapter 15: Using and Reusing Data. Priscilla Kaplan et al., Towards Interoperable Preservation Repositories: TIPR, International Journal of Digital Curation, Issue 1, Volume 5 (2010) TIPR website, Putting It All Together: Creating Preservation Workflows Rainer Schmidt, et al., A Framework for Distributed Preservation Workflows, The International Journal of Digital Curation, Issue 1, Volume 5 (2010): Chien-Yi Hou, et al., A Scientific Workflow Solution to the Archiving of Digital Media, Archivematica: Creating a Comprehensive Digital Preservation System, Ross King, Tools: How to Integrate the Components of Digital Preservation, Case Studies of Preservation Workflows Planets, Guidelines for Creating and Installing IF Preservation Workflows and Templates, D1_Creating&Install_IF_Pres_Workflows.pdf IIPC (International Internet Preservation Consortium) Workflows WorkPackage: Digital Preservation in the Web Archiving Workflow Workflow Models, Online Activities Weekly Blog o Topic: Preservation Workflows Using some of the examples that you have been examining in the last few weeks (both in the readings and in class) as your guide, think about how you might construct a workflow for a particular transformation, such as migrating a file from one format to another, or how one might create an access copy of a file from one stored in an AIP. Try to be as detailed as possible, breaking down the procedure into the smallest steps possible and indicating what tool could be used for each action. 12

13 Week 4 (November 15-21, 2010) Please consult Vista for preparatory activities to be completed prior to our class session. F2F Meeting: Saturday November 20, 2010, 9:00 AM 5:30 PM, 310A Library Morning (9:00 am noon) Assessing Digital Preservation Needs Preservation Planning and Administration Readings Digital Curation Centre, Data Management Plan Content Checklist, Northeast Document Conservation Center, Planning for Digital Preservation: A Self-Assessment Tool, nal.pdf Pauline Sinclair, et al. Are you Ready? Assessing Whether Organisations are Prepared for Digital Preservation, paper presented at ipres 09, San Francisco, 2009, Case Study: Knowledge Management Working Group Webinar Archive (Guest: Dr. Denise Bedford) Other Readings and Resources TBA Lunch (12:00-1:30 pm) Afternoon Laboratory (1:30-5:30 pm): Introduction to PLATO (Preservation Planning Tool) Readings and Resources PLATO: The Planets Preservation Planning Tool, Christoph Becker, et al., Plato: A Service Oriented Decision Support System for Preservation Planning, JCDL 08, June 16 20, 2008, Pittsburgh. 13

14 Christoph Becker, Digital Preservation: How to Plan, Team Digital Preservation and the Arctic Mountain Adventure, Activity (To Be Completed After F2F Meeting) Assignment #2, due November 29, 2010: o Preservation Planning Case Study (Knowledge Management Working Group Webinar Archive) Week 5 (November 22-28, 2010) Online Happy Thanksgiving! Migration and Emulation Tools Readings Paradigm: Workbook on Digital Private Papers, Selecting the Right Preservation Strategy, Migration, Paradigm: Workbook on Digital Private Papers, Selecting the Right Preservation Strategy, Emulation, Christoph Becker, et al., Systematic Characterisation of Objects in Digital Preservation: The extensible Characterisation Languages, Journal of Universal Computer Science, vol. 14, no. 18 (2008), Jeffrey van der Hoeven, et al. Emulation for Digital Preservation in Practice: The Results, International Journal of Digital Curation, Issue 2, Volume 2 (2007): Kam Woods and Geoffrey Brown, Assisted Emulation for Legacy Executables, International Journal of Digital Curation, Issue 1, Volume 5 (2010): Christoph Becker, et al., Keeping the Game Alive: Evaluating Strategies for the Preservation of Console Video Games, International Journal of Digital Curation, Issue 1, Volume 5, 2010: René van Horik and Dirk Rooda, MIXED: Repository of Durable File Format Conversions, ipres 2009 proceedings, San Francisco,

15 Resources Dioscuri, Universal Virtual Computer, Software Preservation Society, Walker Sampson, An Annotated Bibliography: Approaches to Software Preservation, Online Activities Weekly Blog (Topic: TBA) Assignment #3, due December 6, 2010: o Evaluation of Emulation Software see instructions on Vista Week 6 (November 29-December 5, 2010) Please consult Vista for preparatory activities to be completed prior to our class session. F2F Meeting: Saturday, December 4, 2010, 9:00 AM 5:30 PM, 310A Library Web Archiving Tools Harvesting and Access Morning: Case Studies Case Studies: 1. Heretrix/Archive-It (Internet Archive), / 2. Web Archiving Service (California Digital Library), 3. Content DM Web Harvester service (OCLC), Readings: Alex Ball, Web Archiving, Digital Curation Centre, March Adrian Brown, Archiving Websites: A Practical Guide for Information Management Professionals, Facet, Read Chapters 1, 2, 4, 5, 7, Appendices 3, 4, 5. 15

16 Markus Enders, A METS-Based Information Package for Long Term Accessibility of Web Archives, ipres 10, Vienna, Austria, Tracy Seneca and Shifra Pride Raffel, NDIIPP Web-at-Risk: The Development of a Web Archiving Service at the California Digital Library, Afternoon: Laboratory Harvesting/Organizing/Accessing Activities We will be taking a look at a few tools for harvesting any tools that we don t look at this afternoon will be candidates for your Web Harvester Tool review assignment (see below). Assignment Assignment #4: Review of Web Harvester Tool, due December 13, 2010 o Write a review of one of the web archiving tools listed below (choose one that we did not look at in class on December 4, 2010). Depending on the tool, you may need to download a program locally, access it online, or use the tool at the laboratory. NetarchiveSuite, HTTrack Website Copier, GNU Wget, (for students comfortable with command-line apps) Web Copier, (commercial software; download free trial version) OCLC Digital Archive (commercial service; instructor will provide instructions for access) Web Curator Tool, (will be available at Kent SLIS laboratory) Week 7 (December 6-12, 2010) Online Web Archiving, pt. 2: Curatorial Objectives and Intellectual Property Concerns Readings: Adrian Brown, Archiving Websites: A Practical Guide for Information Management Professionals, Facet, Read Chapters 3, 6, 8, 9, 10, Appendix 2 16

17 June M. Besek, et al., Digital Preservation and Copyright: An International Study, International Journal of Digital Curation, Issue 2, Volume 3, 2008: Katie Scott, Archiving Britain s Web: The Legal Nightmare Explored, Wired.co.UK, March 5, Web Archiving Service, Information for WAS Curators, (browse through guidelines, forms, and tutorials) Resources: Web Curators Mailing List, Online Activities Weekly Blog Topic: Subscribe to the Web Curators Mailing List for at least a week (see above link for instructions on how to subscribe) and report back on a topic of interest relating to harvesting, indexing, and providing access to webbased material. Week 8 (December 13-17, 2010) Online Looking Ahead: The Next Generation of Digital Preservation Technologies Readings David S.H. Rosenthal, Bit Preservation: A Solved Problem? International Journal of Digital Curation, Issue 1, Volume 5, 2010: José Barateiro, et al., Addressing Digital Preservation: Proposals for New Perspectives, InDP 09, June 19, 2009, Austin, TX, Other Readings TBA Online Activities Weekly Blog Topic: Identify a critical need in digital preservation technology and report on progress towards that goal. Use current literature, including blogs, conference presentations, and journal articles to document your topic. Final Assignment, due December 17, 2010: 17

18 o Web Archiving Project (instructions available on Vista) 18