The Australian War Memorial s Digital Asset Management System Abstract The Memorial is currently developing an Enterprise Content Management System (ECM) of which a Digital Asset Management System (DAMS) will form part of this implementation. The DAMS is central to the sustainability of our National Collection digital assets. The new DAMS will manage over two million preservation and access images of records as well as over 250,000 digital objects including digital born collections, photographs, film and sound recordings. Our current storage capacity for digital assets is at 2 X 50 TB. The second 50 TB is for backup and our storage requirements are growing. The DAMS integrates with existing collection management databases, provides digital asset ingestion and deployment including bulk import of assets and metadata, capture metadata, manage and preserve digital assets, and has automated asset processing. The DAMS also manages security considerations through permissions corresponding to assets and folders containing them. Interwoven MediaBin was used as base infrastructure for this system. The implementation of the DAMS required customisation for our particular needs and this was designed by Alphawest and the Australian War Memorial. This presentation will focus on what the Memorial required and how we achieved our DAMS solution. Presentation The Australian War Memorial is custodian to one of Australia s most significant museum collections. Over four million items record the details of Australian s involvement in military conflict, from colonial times to the present day. A visit to the Memorial s website will reveal over two million pages of digitised records available through various databases as well as around 250,000 photographs and digital images of various collection objects from planes to art images. For over 10 years the Memorial has engaged as core business the digitising of the National Collection as a preservation activity. Digital preservation is the strategic approach that underpins the Memorial s National Collection preservation principles. Some items are recorded as original digital assets and can not be replicated from any analogue sources. Digital technology is used to preserve the information content of deteriorating originals such as images and audio. Other initiatives include the digitising of fragile items such as paper based items, including maps, documents and the art collection. Digitisation is part of the Memorial s process to not only preserve original collections but also enhance public access to collections. It is also part of the Memorial s copying service. The Memorial digitises access copies of archival collections and provides them to the public on disc for reference purposes. A growing proportion of the Memorial s collections are now born-digital in that they have originated in a digital form as opposed to assets that have been digitised from analogue collections. The largest proportions of born-digital items held at the Memorial are in the photograph and film collections. Most people have digital
cameras and are familiar with creating and managing these items in the digital environment. However, a growing number of born-digital office documents and emails are also coming into the collection. The Memorial is receiving increasing donations of electronic manuscripts, databases and born-digital maps reflecting Australia s operations in current conflicts. We are also generating our own born digital files in Sharepoint and these official records will eventually form part of the National Collection and be managed in the DAMS. Born-digital material is the most problematic for long term preservation. It comes into the collection on all manner of carriers that the Memorial may or may not support, with risk of virus and in formats that we may need to standardise. In line with the development of the DAMS the Memorial has also developed new and enhanced existing policies and workflows to manage these emerging collections. The Memorial s digital collections grew over the last decade without a permanent functioning DAMS to manage and store digital assets. Collections were stored on a shared drive in a variety of places and backed up on tape. Digital-born objects, high resolution preservation images, low resolution access images for the web and for public reference all required management and a long term preservation solution. The Memorial needed to ensure that these digital resources were preserved, protected, managed and maintained for their long term viability and integrity. In the first instance the Memorial began investigating a DAMS to manage workflow, storage, preservation and display of our large digital preservation programs and emerging digital-born collections. Digital preservation involves a number of linking technical and management elements such as object validation, format registry, checksum, virus control, metadata management, secured storage and disaster management. We needed to facilitate access to assets for the web and a system that could deploy to the web and to the collection management systems. The DAMS needed to service and operate with our major corporate systems. The Memorial does not have an existing centralised catalogue system that can manage and support images for all of its collections. The collections including the digital collections are catalogued on several different management systems. The Memorial is a museum (MICA Collection Management System 1 ), a library (FIRST books database 2 ), and an archive for Commonwealth records (RecordSearch 3 ). RecordSearch is a shared database between the Australian War Memorial and the National Archives of Australia. Over the last decade to support and control images related to collections the Memorial created yet another collection of separate databases to provide biographical data as well as access to digitised archival and book collections 4. We call these the Research Centre databases (RCDB). Documentation relating to the collection is also held in a variety of places and formats including extensive written records, data in electronic information systems including our internal and external databases and in digital born files on the Memorial s servers. Memorial staff need to access information on item identification, provenance, location status and copyright in order to facilitate access to the collections for loans, exhibitions and sales. A search application became part of what was needed to be developed to assist users to browse, discover and find what they want across our entire collection and all catalogues. 1 www.awm.gov.au/search/collections/ 2 www.awm.gov.au/collection/books/ 3 www.naa.gov.au/collection/recordsearch/ 4 www.awm.gov.au/research/people/ www.awm.gov.au/collection/war_diaries/
The Memorial provides online access to a large amount of content, but it is through static pages or via our museum content management system 5. Some of our existing CMS s do not hold or facilitate the display of images. The library management system, for example, does not support images and we can only display digitised books, maps and other published digital collections via an html link to images stored and displayed elsewhere on our website. Our shared RecordSearch database also does not support Australian War Memorial collection images and is also not exposed to Google making searching limited to accessing the database. It was hoped that the development of an ECM would provide some better solutions for the display and access of images. To accommodate and consolidate the Memorial s growing digital collections, improve record keeping and streamline the website's content. The Memorial decided to overhaul its content management systems and embark on a project to facilitate Enterprise Content Management. To avoid the risks associated with managing any major software development internally the Memorial looked for an off-the-shelf solution which could integrate with our existing applications. Scoping of the project occurred in 2007. Phase 1 of the ECM project provided detailed analysis of corporate needs and delivered a business case which lead to approval of the concept and funding. Phase 2 saw completion of the development of detailed specifications. Interwoven's (now Autonomy) MediaBin was selected for digital asset management. With the partnership of Alphawest, the project included the replacement of inhouse developed systems which were unable to scale, meeting our storage needs and to accommodate the increased application of digital video and sound. With only a small IT team the Memorial did not want to support and manage multiple applications and handle major customisation. Therefore we sought to minimise the number of products and interfaces users utilise. This solution was also to integrate with the Memorial's various library management packages and in-house databases. ECM project objectives: Preservation based digital asset management Electronic document management Access to corporate and public information Project components: DAMS Digital Asset Management System Autonomy MediaBin EDRMS Electronic Documents and Records Management System Trim with Alphawest enhancements and Sharepoint WCMS Web Content Management System - Drupal 6 Enterprise Search Idol Retina 7 with Alphawest enhancements DAMS Design The DAMS component of the ECM has been implemented as a management process for the life cycle of the Memorial s digital assets. The DAMS consists of management tasks and decisions surrounding ingestion, preservation, annotation, cataloguing, storage retrieval and distribution if digital assets. 5 For example: http://cas.awm.gov.au/item/pr04498 6 http://drupal.org/ 7 http://publications.autonomy.com/content/results/index.en.html?query=summarization
The core business tool within the DAMS is MediaBin, this application manages the preservation and technical metadata of each asset. MediaBin is not integrated with the existing CMSs, however at certain stages of the ingestion process some metadata is exchanged. The CMS - MICA, RCDB and FIRST continue to be the authority systems that manage the National Collection for rights management (including copyright and reproduction rights), locations, values, contextual information and delivery to the Web. Corporate users use MediaBin to manage corporate assets. Implementation of a DAMS-CART will allow all Memorial users controlled down load of collection assets. All assets within MediaBin are preserved to the same level. Corporate users can ingest corporate images, for example, photographs of an event at the Memorial but will be required to record and complete a base level of metadata on ingestion to the DAMS. Staff can search the DAMS for reference versions of National Collection and Corporate assets. The DAMS will deploy images to MICA and RCDB and these CMS s will control what assets will be published to the web pages. MediaBin design MediaBin is designed around the concept of a Producer Area and Library area. The Producer Area is a work-in-progress space where digital assets are initially uploaded into the MediaBin. Metadata is captured or recorded and quality assurance occurs here. The assets here are not visible to most staff members and need to be promptly deposited in the library. Assets in the DAMS are controlled by security guidelines that enable only limited users to conduct some of the tasks. Digital assets from the Producer Area are deposited in the Library after they have passed the preservation and business rules process checks. The Memorial and Alphawest designed the requirements and code for these business rules and preservation processes. The library area is a read only repository for finalised digital assets. Most assets here can be seen by most staff. They are linked from the existing collection management systems (MICA/FIRST/RCDB). These digital assets can also be found via systems like Federated Search.
The DAMS also required some networked workspaces where assets are placed before deposit and also retrieved to manipulate and use. These are network folders with a limited size. Items for publication, exhibition and sales are retrieved from the library area into these network folders. Assets only remain here while the asset is manipulated and utilised. The original asset is always stored within MediaBin. Assets can be deposited into the DAMS via automated ingestion processes facilitated by hotfolders or by manual insertion tasks. Users navigate to his/her respective folder in the Producer Area and upload asset/s utilising one of the specific insertion tasks related to their purpose or role. Users add and QA metadata on selected assets in the Producer Area and select asset/s to initiate the Deposit process. The DAMS checks, assesses and validates assets against specified business rules including Checksum (MD5), file format, and technical metadata. Assets that pass the business rules are deposited into the Library. Those that fail are given a log detailing the nature of the failure. Asset/s that pass the checks are updated with additional metadata (manual / automatic) including integrated metadata from the CMSs (MICA, RCDB or FIRST). Assets in the Library can be updated. Those that need revising can be moved by authorised users back to the Producer Area. They are checked-out, modified, checked-back in to the producer and then re-deposited into the Library. This will generate versions of the asset within the system. The older versions are always preserved. This has the potential of blowing out our storage costs and therefore checking for errors is vital prior to deposit. Users are able to search, preview and download the desired asset/s via specific retrieval tasks. Authorised users can retrieve or email MediaBin assets (as shortcuts or attachments). Optional transformation tasks can be done while the asset is being
retrieved including changing the size and format. Automatic deployment of selected collection assets occurs for use on websites via the CMSs. Access to Mediabin is via a web browser. Staff access the URL for the MediaBin Web client and enter their User ID and Password. The three main tasks related to assets in the DAMs are insertion, transformation and retrieval. Insertion tasks are those that modify the preview image or add metadata to the ingested asset. Transformation tasks are those that make asset changes including, size, resolution, colour space and so on. Retrieval tasks allow download of original assets rendered in formats for particular usage, for example, PSD - JPEG. The DAMS provides some automated workflows which were required for bulk ingestion of images and image ordering and these were developed as part of the system. All assets that are to be stored in the MediaBin Library must have a record created in the existing CMS (MICA, RCDB or FIRST) before ingestion. The CMS data exchange attaches the image to the CMS record and returns metadata to MediaBin including accession number. Preservation The Memorial has adapted the PREMIS 8 Digital Preservation Frame to ensure that the DAMS solution is scalable and sustainable. The Memorial s digital preservation principles include ensuring that digital collection assets and systems are constantly monitored to ensure the integrity of the bit-stream data. The DAMS enables management of issues such as degradation, obsolescence, corruption, security and migration. The DAMS deals with preservation metadata. MediaBin also keeps contemporaneous records of all pertinent actions. The DAMS has a transaction audit process and records of actions and administration processes that are relevant to archival preservation. The repository has mechanisms in place for monitoring and notification when representation information (including formats) approaches obsolescence or is no longer viable. Preservation checks occur when ingesting and within MediaBin. The JHOVE program is used to validate the file format - a healthy asset will get a value of "Well- Formed and valid". If a file is corrupt it will not ingest into MediaBin. Secondly the file's MD5 checksum is calculated. The system includes ongoing checksum comparison processes and reporting/correction and system backup/recovery procedures. Back-up to tape provides offsite protection against degradation and potential loss of assets. In conjunction with the development of the DAMS the Memorial has developed policies and workflows to manage the diversity of digital formats and format redundancy that preserves accessibility of asset types. Restrictions The Memorial holds restricted items across the collection due to the Archives Act, donor access agreements and embargoed items. Access permissions within 8 http://www.loc.gov/standards/premis/v2/premis-2-0.pdf
MediaBin are set in folders, tasks, metadata and metadata groups. Access to restricted items is limited through user s login and their membership in user groups. Storage solution One of the core functions of the DAMS was the system must be highly scalable and sustainable to manage the growth rate and volume of digital assets in the collection. The DAMS employs a (duplicated) storage strategy to prevent catastrophic loss of the unique assets. The project required 100 Terabytes of storage for digital media over the next three to five years. Disaster recovery copies will be on tape off site; this is also part of our storage solution. Training The Memorial used Implementation Smart Assistant or SmartAss (as it is sometimes called) guides to develop some training resources and courses. This system provides users with visual performance support that is contextual. As you are using MediaBin the Media Guide demonstrates the functionality you are currently using in the system. MediaBin will also be supported with detailed SmartAss interactive training guides through the program, plus there are printable versions and manuals made available. We also employed end user training to all staff who would be using the system. We commenced with the power users, those staff that will be using the system directly to ingest and preserve assets. Training for implementation was divided into two levels of support. Throughout the Memorial a number of staff were trained to conduct most of the end user training and support new users. These people are the first level of contact for problems. Level two support will enable staff to either log a problem with our IT Section via a web form or helpline. We have created a DAMS User Group (DUG) consisting of National Collection ingestors (people who digitise the collection or create digital objects about the collection). This group includes preservation scanner operators, photographers, film and sound preservation officers, conservators, nominated curators and digital image operators. Evaluation This was a major and challenging project for the Memorial, but some relevant past experience in introducing successfully separate information systems assisted in the development and management of the project. The implementation of the DAMS has facilitated the update and development of policies and workflows to manage digital collections, manage rights and the diversity of digital formats and format redundancy that preserves accessibility of asset types. While the DAMS can manage restricted material copyright was more of a problem. It is more changeable and because there was a time lag between the CMS and the DAMS it was decided to take the rights management out of the DAMS and staff are to consult the CMS for issues regarding rights. The DAMs is currently not able to manage security classified records, these are held and managed within the creating agencies at present.
Overall the DAMS achieved its objectives on its delivery in December 2010. However, the process drew in a lot of resources including staff time across the Memorial. It especially tied up the time and resources of our IT Section for over three years and some other important projects have suffered as a result. The Memorial ended up doing a lot of its own customisation and will need to do a lot more in the next two years. Some collections included in the ECM scoping fell outside of the system and some important digital collections are still without a DAMS solution post implementation. These will require work by the Memorial s IT Section to develop code and processes for their long term management in the DAMS.