Federal Agencies AV Working Group

Transcription

1 Federal Agencies AV Working Group Notes from the meeting held in conjunction with the IASA/AMIA joint annual conference in Philadelphia, November 1, 2010, with additional notes from related subgroup meetings. Abstract: Meeting with representatives from organizations in ten nations plus the United States to discuss plans for developing an MXF Application Specification for audio-visual content. The notes for this meeting include summaries of discussion on the same topic from additional meetings of a subgroup of Audio-Visual Working Group of the Federal Agencies Digitization Guidelines Initiative. Introduction This document reports on a meeting that discussed a proposed MXF Application Specification for Archiving and Preservation (MXF AS-AP). The November 1, 2010, meeting was convened by the Audio-Visual Working Group of the Federal Agencies Digitization Guidelines Initiative at the IASA/AMIA conference. This document will also summarize some of the key outcomes of meetings prior to and following the November 1 event that brought together an ad hoc subgroup of the AV Working Group with a special interest in MXF AS-AP. Background: specs and shims MXF is a standard from the Society of Motion Picture and Television Engineers that defines a container or wrapper for audiovisual content. MXF application specifications document a set of constraints on the use of this container and the essences they contain, e.g., uncompressed picture information or JPEG 2000 frame images. Application specifications may also place constraints on the handling of metadata, file "architecture" (structure), the treatment of closed captioning, and so on. For more information, see the Working Group's background paper: AV_AppSpecProj_Bkgd_ pdf Although an application specification constrains the use of MXF, it will not provide the narrow, specific cookbook that an organization requires for a particular category of content. If application specifications were that narrow, the community of users would be required to develop dozens of similar documents. Therefore, developers write specifications that are "somewhat broad," and then add appendixes--as many as may be required, over time--that further constrain the specification and thereby provide the needed detailed cookbooks. These further constraints (the appendixes) are called shims. Summary of discussions at pre-meetings The ad hoc subgroup met in July, August, and September 2010, and again in December 2010 and January The meetings in July through September yielded the working version of AS-AP that was discussed at the November meeting. In addition to the background paper cited above, a copy of the draft specification was posted at the Federal Agencies Web site: pdf 1

2 Both the pre- and post-november meetings highlighted some issues pertaining to the scope for the desired specification and its shims. Regarding breadth of coverage for the specification proper. All parties would like to have a specification or a closely related set of specifications that cover a broad range of audio-visual content: (a) reformatted from videotape, (b) reformatted from motion picture film, (c) reformatted and born-digital audio, and (d) born-digital moving image content. Can one broad specification cover the full range or is it wiser to draft a set of parallel specifications? The technical leaders of the effort have consistently favored a single, broad application specification, although this will be re-examined intermittently as the drafting proceeds. Regarding the scope for shims. The discussion highlighted a number of ways in which shims might be defined. If the main application specification were broad, we might want (a) a shim for files made from videotapes, (b) a shim for files made when film is scanned, (c) a shim for files created when incoming born-digital moving image content was re-packaged or transcoded. There were also concerns to be sure that the set of shims accommodated such things as (i) audio-only files, (ii) content originally produced in, say, Europe at 25 fps and/or for PAL video systems, and (iii) legacy content--mxf files produced in very recent years by equipment on the market prior to the publication of AS-AP. These latter entities may or may not require shims of their own. The question will be part of the ongoing deliberations. The pre- and post-meetings also highlighted the value of cooperative work with the Advanced Media Workflow Association (AMWA). 1 The incubation of MXF application specifications is the special province of the AMWA and the organization provides a meeting ground for users and vendors. The Library of Congress has joined the AMWA on behalf of the Federal Agencies effort, and plans to work with its Technical Steering Committee to develop the specification. Meanwhile, it is worth noting that Library of Congress funding and human resources to support this activity are limited; thus the overall pace of development will be measured rather than rapid. It would be very helpful to attract participation by persons with engineering knowledge as we address a number of technical questions. Interested parties should contact Working Group coordinator Carl Fleischhauer (cfle@loc.gov). Attendees at the November meeting The meeting space and arrangement for the November meeting in Philadelphia were made with the generous cooperation of the International Association of Audiovisual and Sound Archives (IASA) and their technical committee. There were 51 people present from the United States and ten other nations, including five Federal Agencies Working Group members and three consultants. The group was weighted toward public archives but a number of private sector companies were also present, welcome participants since the future adoption of the application specification will depend upon tools from vendors as well as via open source means. Carl Fleischhauer and Jimi Jones (also on the Library of Congress Federal Agencies team) organized the meeting, chief consultant Oliver Morgan led the main discussion, and AMWA executive director Brad Gilmer also made a presentation. 1 Home page: Application specification page: 2

3 Institutions represented: Australia, National Library Australia, National Film and Sound Archive Austria, Phonogrammarkiv Austria, Vienna Mediathek Brazil, Carlos de Andrade (audiovisual engineer) Canada, Libraries and Archives Canada Canada, Vancouver City Archives Germany, Albrecht Haefner (consultant) Italy, Eurix Group (private sector company) New Zealand, National Library Norway, National Library Slovenia, Institute of Ethnomusicology UK, BBC US, Front Porch Digital (private sector company) US, Academy of Motion Picture Arts and Sciences, Science and Technology Council US, Advanced Media Workflow Association US, Ascent Media (private sector company) US, Audiovisual Preservation Solutions (consultants) US, AVAN project (non-profit) US, Harvard University US, Indiana University US, Langdell Creative Broadcast Services (private sector company) US, Library of Congress US, Media Matters (private sector company) US, Merrill Weiss (consultant) US, Metaglue (consultants) US, National Archives US, New York University US, Oklahoma State University US, SafeSound Archive/George Blood Audio (private sector company) US, Smithsonian Institution US, Utah, University of US, Witness (human rights advocacy) Summary highlights from the meeting Virtually all attendees stayed with the meeting for its full three hours, from 3 to 6 pm. This fact, and a general reading of the flow of questions and comments from the group, led the Federal Agencies team to conclude that the response was favorable to the idea of developing an MXF application specification for archiving and preservation and we are encouraged to continue this work. The November meeting was intended to introduce and explore the topic and the discussion highlighted a broad range of subjects, from technical issues to licensing. 3

4 Speaking as both a Working Group member and as the technical lead for the Library of Congress Packard Campus for Audio-Visual Conservation, James Snyder discussed the Library s commitment to MXF and JPEG 2000 lossless-compressed video for long-term preservation. Snyder reported that this form of compression was providing an average of 2.3:1 reduction in file size; thus, standard definition file sizes drop from about 120 GB/hour to a little more than 50 GB/hour. This Library of Congress unit plans to put all of its audiovisual content into one standardized format to maximize the feasibility of future migrations and support "making content recoverable in hundreds of years. The representative for another federal agency presented a contrasting perspective. In their work today, this agency makes AVI files that wrap uncompressed video. This approach is partly due to the lack of available tools to produce, inspect, and play back MXF and lossless JPEG 2000 moving image picture content. It also reflects the agency's view that its video objects are relatively stable and fixed once they land in long-term storage. Nevertheless, meeting attendees were told, this agency will leave this question open, seeing the potential merit of an MXF approach in the future. Thus they see great value in the effort to define MXF AS-AP and they will continue to participate in the project. There was some discussion of metadata and MXF. One aspect concerned the possible fit with the Metadata Encoding and Transmission Standard (METS). Although there are currently no METS adopters in the Federal Agencies Working Group, a few others in the meeting use METS. They asked, "Are METS and MXF complimentary or competitive?" This specific question was not answered, but one federal agency representative offered a perspective from a non-mets-using organization. He argued that generating METS records for moving image objects would add a potentially costly (in terms of effort) processing step that could be obviated by simply bundling metadata into the MXF file. Another metadata-related comment pointed out that the current plan for AS-AP calls for a minimalist "header" as required metadata. The current plan also allows for organizations to add variable, "local option" metadata in the package (file), to be labeled in an appropriate way. This permits such additions as further descriptive metadata supplied by an enthusiastic curator, or varieties of specialized technical metadata. In the most informal way possible, Oliver Morgan took straw votes on two questions regarding current practices and/or future preferences. Regarding picture encoding, he asked for raised hands about uncompressed streams (many hands) as compared to JPEG 2000 frame images (very few hands). For born digital picture, he asked, how many planned to retain the encodings "as acquired" (several hands). Regarding the encoding for soundtracks, he asked what people considered working with: PCM (many hands), multichannel MPEG formats (some hands), and retain-as-acquired (several hands). Although the straw votes touched on the matter of picture encoding preferences, the meeting did not include a comparison of encodings. This was a disappointment to some attendees who wished for more information on this important topic. One European representative, for example, referred to his archive's use of the FFV lossless codec, developed by the FFmpeg open source software project, saying that he would like to discuss this with other knowledgeable persons. One attendee asked AMWA executive director Brad Gilmer about intellectual property and licensing that may pertain to the MXF AS-AP. There was not time at the meeting to fully 4

5 explore this topic. Following the meeting, Carl Fleischhauer and Gilmer exchanged information about this subject, and the Working Group will send around more complete information soon. For now, suffice it to say that all parties agree that there should be no licensing or restrictions on the adoption and use of the application specification. As will be reiterated in the longer message to come, the AMWA also reminds users that any AS will reference pre-existing standards like MXF and JPEG 2000 which may contain patented elements on their own. In a sidebar at the meeting, the independent consultant Merrill Weiss presented some information about the SMPTE AXF standard-development project. AXF stands for the Archive exchange Format, and it is intended to address the special requirements of digital moving image data (often in very large files) in relation to data storage media. This is an interesting development but one that does not bear in a direct way MXF AS-AP. What's next? At the November meeting, the organizers expressed their goal of re-convening the interested parties in January 2011, via teleconference, for a continuation of the discussion. This meeting was to be preceded by the distribution of the "next draft" of MXF AS-AP. At this writing, there is not yet a next draft. Meanwhile, at the AMWA, some actions have taken place to set up a technical group but it has not yet really started its work. More information about this will be sent soon. Meanwhile, we are deferring the planned teleconference until February or March. 5