Two Words, Two Challenges: distinguishing Audit and Certification of digital archives Hans Hofman, Seamus Ross, Perla Innocenti, Raivo Ruusalepp, Andrew McHugh DLM Forum, Toulouse, 11 December 2008 Digital Curation Centre (DCC), DigitalPreservationEurope (DPE), HATII at the University of Glasgow & Nationaal Archief Netherlands DRAMBORA 1
Overview Situation Audit & certification what do we need? Risk assessment DRAMBORA: scope and role lessons learned Summary 2
Objective of digital longevity Digital preservation aims to ensure that future users will be able discover, retrieve, render, manipulate, interpret and use digital information in the face of constantly changing technology It involves conservation, renewal, restoration, selection, destruction, enhancing, updating, and annotating It is a risk management activity at all stages of the longevity pathway It is about translating uncertainties into manageable risk However: is this the whole picture.? Shouldn t it all start at the creation (or even design) stage? In the digital age we are all digital creators whether in our work, in our community or in our personal life 3
Let s Consider Repositories ERPANET, 2002 ERPANET, 2004 Museums (Vienna) Archives (Wellington NZ) 4
Digital Repository Increasing range of content collections are referred to as repositories Widespread use of term goes hand in hand with diversity of meanings in different contexts digital libraries, research, learning, e-science, publishing, records management, archives, In real life, not all repositories are alike Within different communities the motivation for creating repositories differs and the key services they may provide range over many functional areas Not all of them even aim to preserve the content they are holding 5
Some Challenges What repository definition or implementation models may be rightfully defined as trusted digital repositories? (e.g. are those based on OAIS the only ones we should accept?) What are the attributes and functionalities of a trusted digital repository? How are the concepts of reliability, authenticity and trustworthiness interpreted in different contexts and why? What impact does this have on design? How do we know that our information is reliable? What are the roles and responsibilities of the different stakeholders? How can they be addressed? How can content creators be motivated to participate as well as to use them? How do different communities see trusted digital repositories? How can we create and maintain the trust of user communities overtime and in the face of changing technologies. Reliable conditions vs. reliable objects/information? How to ensure the chain of custody? 6
Repositories must. Ensure stuff ingested into the archive can be output (e.g. be accessible) Handle a wide array of digital media types Guarantee authenticity of the objects they hold Protect Integrity (from intended and untended harm) Enable verification Be secure Maintain all documentation in-house Have disaster recovery functionality built-in Have exit strategies In addition. 7
be trusted Processes: Workflows Operation (management of integrity, authenticity, intelligibility, and accessibility Automation (e.g. ingest, management, publication) Documentation of procedures Auditability Architecture and Implementation People Organisation..[and more] 8
Digital preservation repository core criteria An intellectual context for the work: Commitment to digital object maintenance Organisational fitness HATII UofGlasgow, 2007 Legal & regulatory legitimacy Effective & efficient policies Acquisition & ingest criteria Integrity, authenticity & usability Audit trail and metadata Dissemination Preservation planning & action Adequate technical infrastructure 9
Audit challenges What do we want to achieve or pursue with audit? What should be audited (e.g. repository level, data set, capability) in what context? Who should do the audit (e.g. specialised bodies or not)? What are the requirements for auditing organisations? What framework(s) do we need in relation to the different business contexts to conduct an audit? What processes and steps are necessary to conduct a proper audit? and what steps should the audit process encompass? Should an audit be followed by (deliver) certification? 10
Let s think simple ١. Document what you think or say you do ٢. Be able to demonstrate that you can do what you say ٣. Be able to show that you do do what you say ٤. Make colleagues aware of what you do and what their roles are Test to see (1-3) 11
Required Need to describe evidence base to contribute towards consistency to create a mechanism that ensures conclusions can be validated practical applicability depends on identification of objective means to demonstrate compliance efforts must probe for evidence of concrete processes, structures, and functionality documentary, testimonial and observational evidence Need to establish preservation pressure points including uncertainties and risks risk awareness is low within the community 12
Digital Repository Audit Method Based on Risk Assessment (DRAMBORA) A trusted digital repository will understand threats to and risks within its systems. from the introduction to the TRAC Criteria & Checklist Developed by DCC & DPE, DRAMBORA encourages repositories to: develop an organisational profile, describing and documenting mandate, objectives, activities and assets; identify and assess the risks that impede their activities and threaten their assets; manage the risks to mitigate the likelihood of their occurrence establish effective contingencies to alleviate the effects of the risks that cannot be avoided. 13
DRAMBORA offers an organisation context-aware process for repository assessment self-audit that repositories do themselves, based on the provided tools assessing capabilities and identifying weaknesses and strengths how well is the repository managing the risks it is facing when it does what it does? suitable for various maturities flexible and valid for repositories of all shapes and sizes and of different contexts advice on how to overcome the risk situations and what other repositories have done in similar situations Methodology, tools and associated examples support: Validation [ Are my efforts successful? ] Preparation [ What must I do to satisfy external auditors? ] Anticipation [ Are my proposals likely to succeed? ] 14
Risk Management Model IDENTIFY INTERNAL AND EXTERNAL CONTEXT IDENTIFY RISKS MONITOR AND REVIEW COMMUNICATE ANALYSE AND ASSESS RISKS MANAGE AND TREAT RISKS 15
Outcomes and results Following the successful completion of self-assessment, organisations will have: established a comprehensive and documented self-awareness of their mission, aims and objectives, and of intrinsic activities and assets constructed a detailed catalogue of pertinent risks, categorised according to type and inter-risk relationships created an internal understanding of the strengths and shortcomings of the organisation provided the organisation with a tool for continuous management of risks prepared the organisation for subsequent external audit 16
DRAMBORA Interactive www.repositoryaudit.eu 17
DRAMBORA Workflow Preliminary collecting and analysis of repository documentation Organize appointments and onsite visits with repository staff (managers, curators, IT, legal experts ) Risk registry finalization Audit report finalization Impact on individuals and organisations 18
Pilot Assessments 19
DRAMBORA: Present 80 or so example risks to prompt thinking... insufficient DRAMBORA Interactive enables repositories to align their objectives, activities, strengths and shortcomings with other peer repositories' responses To allow comparisons between peer organisations, profiles of repository types need to be developed An attempt at a typical digital library risk profile included in the DELOS report Investigation of the potential application of the DRAMBORA toolkit in the context of digital libraries to support the assessment of the repository aspects of digital libraries Support for peer comparisons should be built into the DRAMBORA Interactive system 20
DRAMBORA collaborates with Trustworthy Repository Audit and Certification (TRAC) Criteria and Checklist Working Group Center for Research Libraries (CRL) Certification of Digital Archives Project Network of Expertise in Long-term storage of Digital Resources (nestor) DELOS Digital Preservation Cluster (WP6) International Audit and Certification Birds of a Feather Group now RAC (Repository audit criteria) SHAMAN (Sustaining Heritage Access through Multivalent ArchiviNg) ISO TC46 /SC 11 Working Group on risk assessment for records systems (based upon DRAMBORA methodology) 21
Lessons learned: risk assessment Risk appears to be an easily understood concept for repositories We have had many discussions with users about the risk impact and probability scores and scales, and have modified them slightly Any risk assessment leaves some room for interpretation keep the purpose of the assessment in mind We have no fixed benchmark on the number of risks or their severity 22
Lessons learned: self-assessment Most of the pilot audits have been facilitated by an expert who has training for DRAMBORA Is improvement in bottom-up self assessment limited by one's own horizons? How can repositories comment on unanticipated risks? When they are unaware of available opportunities? What damage can dishonest auditors do? Are comparability and reproducibility of results compromised? 23
Lessons learned: service classification We want to identify and describe classes of repositories in terms of their common services and characteristics Services are critical, with performance understood in terms of those services Auditors can space their own efforts within the context of comparable repositories They can reflect and inform the perspective of best practice that exists within their own particular 'repository-sphere'. 24
Lessons learned: trust in repositories Strong link between the organisational context of the repository and its users expectations Different focus on preservation in archives and data centres The concept of trust is varying from one user community to another Linking trust to services that a repository is offering is more meaningful than to a whole institution or unit within an organisation 25
Certification what do we certify? is it already possible or realistic given the immature state of digital preservation? what does it prove? what certainty does it provide? will it generate trust? No infrastructure for repository certification yet How will it relate to other certification processes, such as ISO 9000? At the moment DPE has started to train auditors (Prague October 2008, London February 2009, Rome March 2009) 26
What are we working on now? Promotion of the assessment method and toolkit 1208 downloads of the toolkit 2052 downloads of DRAMBORA manual v1.0 78 registered audits DELOS report (2008) DPE Training Programme Development of training materials to support self-assessment (online learning environment (Moodle) Training for general public Training for auditors (Prague 08, London 09, Rome 09) Accreditation of self-auditors Discussion with other working groups developing repository audit checklists 27
Further developments for DRAMBORA Repository profiling Dissemination in international conferences and journals DRAMBORA in Japan DRAMBORA interactive user manual and video tutorial Version 3.0 (downloadable), added visual features, translations in local languages 28
Observations and summary DRAMBORA raises awareness, helps to address issues Different approaches exist not exclusive, but complementary Is auditing repositories the answer to the problem? do we have to look beyond repositories given the current fundamental changes in the web-environment? how to address cross-organisation repositories? interrepository transfer/ exchange/ networks what should be the scope? individual repositories or the web and/or services or all? Are we intervening at the right moment/ place? Still re-active, and not really pro-active approaches. How to ensure the creation of preservable information objects? 29
URLs DCC/DPE Digital Repository Audit Method Based on Risk Assessment (DRAMBORA) http://www.repositoryaudit.eu Trustworthy Repositories Audit & Certification (TRAC) Criteria and Checklist http://www.crl.edu/pdf/trac.pdf nestor Catalogue of Criteria for Trusted Digital Repositories http://www.nbn-resolving.de?urn:nbn:de:0008-2006060703 Ten basic characteristics of digital preservation repositories http://www.crl.edu/content.asp?l1=13&l2=58&l3=162&l4=92 30