Dealing with electronic records: intellectual control of records in the digital age

Transcription

1 Janus Î998.Î Lligall 12 Barcelona Dealing with electronic records: intellectual control of records in the digital age Hans Hofman INTRODUCTION Can you imagine: the archivist as ghosthunter? That is the picture you get when you talk about electronic records. It is like hunting phantoms or ghosts. They are invisible, untouchable and elusive... How are we going to catch them or give them substance such that human beings can handle them? The disappearance of the record as a physical entity is one of the most intriguing and fascinating aspects of the digital era. The familiar paperbased records are being substituted with abstract and in a way invisible electronic records. Paper and parchment records are physical representatives of a past world and can tell us about activities long ago. As objects they give us an idea of how it was, simply because you can touch and even smell them. While electronic records serve the same purpose, they are elusive. If you don't have the right tools or instruments you can't even see them as records, let alone feel or smell them. It has been said many times before that the present generation of records managers and archivists is facing a whole new phenomenon: electronic records. This has been seen by optimists as presenting all sorts of opportunities and by pessimists as a big problem. Both opinions exist. Nevertheless the information society is becoming more prevalent and almost everything seems to turn around data and information. It is estimated that 95% of all information exists in digital form of which more than 10% is now digitally exchanged. In the world of EDI it is even possible to carry out processes automatically without paper or even without any human intervention The question for records managers and archivists is how are they going to manage and control these new electronic records for which they are responsible? An important activity in this respect concerns the intellectual control of records, one of the issues of this parallel session. What does intellectual control mean in a digital world? Or in other words, what impact does the emergence of electronic records have on intellectual control. In the following paragraphs I will discuss some of the main developments in record keeping as consequences of the use of information technology within government administration. I will also examine the role of contextual information with respect to intellectual control, and more specifically with respect to description.

2 134 Barcelona Lligall 12 fanus Finally, I will explore the role that existing archival description standards as described in ISAD/G and ISAAR, can play in this respect? The viewpoint I will take is mainly that of archivists, which are responsible for archival records. DEVELOPMENTS: WHAT IS HAPPENING? It is clear that the dynamic world of information technology has an enormous impact on record creating and record keeping, the area where records managers and archivists are trying to carry out their responsibilities. The consequences are not only technological but also organisational and legal. As mentioned the physical appearance of the electronic record is entirely different from a paper record. It consists of several components including data, software to process the data, hardware, storage medium and documentation. The change in the nature of records is one of the reasons that the existing rules for record keeping, which were designed in a paper-based world, are no longer adequate. Another development is the emergence of networking facilities which have had an effect on organisational structures. One of those consequences is the decentralized storage of electronic records, which makes control more difficult. There is also a growing tendency for information to flow across organisational boundaries, thus making it more difficult to establish the provenance and use of electronic records. This is stimulated by a modern phenomenon which is called chain management, which means that management no longer is limited to one organisation, but encompasses the whole chain of activities of a function to be carried out. This results in a shift of attention or perspective from the organisation which carries out a certain function toward the function which has to be carried out by one or more organisations. Furthermore modern software makes it possible to separate data and contextual information, a phenomenon best reflected in workflow management systems. The purpose of these systems is to manage work processes efficiently. As a requirement of their implementation these systems have a lot of contextual information, such as information about the process and its steps or tasks, about the procedures, about the individuals responsible etc. (i.e. contextual information). These workflow management systems are normally connected to document management systems which store the documents created and used in the process (i.e. the data). What is missing however are the guarantees for authenticity and reliability, particularly since documents or records can be re-used, deleted and altered. Other deficiencies are the difficulty in managing different versions of documents, and the fact that most systems do not support disposition schedules. Another issue which we have to keep in mind is the lack of a systematic and well planned changeover from conventional paperbased record keeping to electronic record keeping. This leads to a lack of knowledge about the records of an organisation.' No one knows what exactly exists on paper or what in digital format. This hybrid situation creates chaos and promotes the tendency to rely for the sake of security on paper even though they were created in digital form. The result is that two information systems exist next to each other: one on paper and one in electronic form.

3 Janus Î998.Î Lligall 12 Barcelona We are facing a transition period which requires things be settled with respect to record keeping especially as it relates to accessibility and authenticity of records. But, without adequate record keeping systems it will be difficult to guarantee these essential requirements for records. If records managers and archivists do not succeed in solving this challenge, then records will not fuflil their role for a government organisation in ensuring accountability providing the memory required to deliver government programs and to make decisions. This could have serious consequences for democratic principles. A complicating factor of these new developments is that everything is changing and that it will be continuous. It is like hunting illusions. We are actually dealing with a 'moving target'. Just when one thinks of having an idea of what is happening, another new phenomenon emerge. The situation is complex and even somewhat chaotic. In summarizing these developments, there are a number of characteristics then. These are as follows: o a record is no longer a physical entity, but physically fragmented, kept only together by a logical boundary o information technology creates new organisational structures, especially because of networking technology; this influences the information flow and raises other requirements for record keeping, o although we are facing a transition period with paper and electronic records existing next to each other; this wil not last, o There will be continuing and rapid change in information technology, which means that change will be constant in our lives. In examining these characteristics a first conclusion should be that there needs to be a shift of emphasis from the physical object to a higher, more abstract or logical level. This will require that careful attention be paid to the domain of intellectual control. What role could this activity play or what are the consequences for intellectual control? HOW TO DEAL WITH THE INTELLECTUAL CONTROL OF RECORDS IN THE INFORMATION AGE? Intellectual control can be seen as the link between the reasons why records are preserved and the records themselves. To put it succinctly: intellectual control focuses on the identification, appraisal, accessibility and understandability of records regardless of their form in order to support the business processes, to guarantee the accountability of a government organisation and to safeguard archival records as cultural heritage. The management of contextual information and the description of records are the main activities that support these activities. Intellectual control starts at the beginning of the life cycle of records and is carried out both in the record creating organisation and the archives. The purpose of both is different. Within the record creating organisations intellectual control is especially aimed at ensuring accountability. Crucial in this respect are the reliability and authenticity of the records, attributes which have been given much emphasis by the research project of the University of British Columbia (UBC) on the integrity of electronic records.

4 134 Barcelona Lligall 12 fanus The record creating organisation is responsible for the reliability of records, (i.e. that 'the record stands for the fact it is about' as the UBC research project defines it). In order to achieve the objectives of authenticity and reliability the record has to be complete. The UBC-project just mentioned has established a template with the necessary elements of a complete record. Authenticity is based on the state in which the records are recorded and preserved and the maintenance of that state through time. The purpose is that the record is not altered, manipulated, or falsified. To ensure authenticity, records should be captured into a record keeping system. The UBC research project contends that in order to stabilize this state of authenticity description is an important instrument. At the moment of capturing a record for preservation, for instance a record profile can be used for each type of record. This is even more important because the physical charateristics of records will change continuously. The only thing that will be steady is the logical record, which exists in a description.^ Other important elements associated with ensuring the reliability and authenticity of records are the classification and registration of the records, and procedures for documentation. The rules and procedures to accomplish these requirements of authenticity and reliability are or at least should be included in the record keeping system. In order to enable the management of electronic records however this record keeping system should contain a description of the record creating business processes. To illustrate hat this means I will give you an example. Since June of last year there exists in the Netherlands a program called 'Digital Longevity' which has as its main purpose to stimulate and support all initiatives within government organisations to realise proper electronic record keeping practices. The program is set up by both the ministry of the Interior and the ministry of Education, Culture and Science. The main approach is to participate in pilot projects initiated by government organisations on all levels. One of these projects is the development of a new information system for the Dutch Parliament, which is being designed to support the activities of the members of Parliament and all the political and legislative processes. It is now in the design stage. The program bureau has been asked to help in defining the record keeping requirements. In doing so, we have learned that the whole context of record creation and keeping needs to be described in order to be able to manage the records and to adapt the system to organisational changes. In fact it appears that several levels of metainformation will be necessary to meet the record keeping requirements At various levels we can distinguish, for instance: description of document types, of types of business processes (or activities/functions), of types of procedures, of the record creating organisation itself. Apart from a description of this institutional environment it is necessary to document the record keeping rules and to maintain a log file of any change or adaptation within the record keeping system. As a result not only the actual data or records are captured, but also a picture of the organisational, its business procedures, which produced them, and a model of the record keeping system in which they are stored. All this is necessary in order to make electronic records 'visible'. Only then can we preserve the records as reliable and authentic, manage them, maintain the record

5 Janus Î998.Î Lligall 12 Barcelona keeping system and account for the changes in the record keeping system that have occurred during its existence. This example should give an idea of the potential kinds of meta-information that are needed. In the concept of a record as described by the ICA committee on electronic records, one of the essential characteristics of a record is that it should 'comprise content, context and structure to provide evidence of the activity' in which it is produced or received. The relationship of the record and the business activity is crucial. A characteristic of the electronic record is that it is recorded information, but since the record has no physical boundaries (unlike a paper record), its boundaries are determined by the business process or function in which it played a role (the individual transaction). It is information on a logical level, which also places a record or a series in its historical perspective: why was it created or produced, what role did it performed, what status did it had etc.? For a government organisation, mformation about provenance (organisation/ record creator and process), procedures, and responsibilities is essential to ensure accountability and good governance. In messages, for instance, the header comprises the contextual information about the working process in which the document plays a role, but this header can easily be separated from the content, the message itself. As a result measures need to be taken to ensure that this contextual information will not be lost. The physical aspects The physical characteristics of electronic records lead to, as already mentioned, a shift of attention in managing records from the physical level in the case of a paper record to the logical level in the case of an electronic record. Paradoxically the electronic record seems to exist merely because there is metadata or contextual information about it. If we don't have that we don't know even that there is a record. Since we can not see the record physically as a separate object, but only temporarily on a computer screen, we could forget it and won't preserve it. In the paper world, sooner or later, we will see the paper record. The electronic record, however, exists only virtually, and can only be presented on the screen which becomes simply the temporary carrier rather than part of the record. The file on a diskette is not the record. This is encoded data which need to be decoded or translated into a record. The consequence of the change in physical characteristics is that we have to describe and preserve also the process concerning how the record was put together. Another consequence is that records need to be described more explicitly in order to manage them intellectually, and in order to guarantee their authenticity, accessibility and reliability. Contrary the administrative control of paper based records, it is necessary to describe the physical characteristics of the electronis record, because they are separated from the logical record. These characteristics include the media on which the files are stored, the (computer) files themselves, and the software necessary to present them. Although it is of no concern for the researcher it is of course necessary to describe the relationships between the logical records and the physical files. Since the physical files will be migrated or converted through time because of technological developments, the management of the relationship will need to be continued.

6 134 Barcelona Lligall 12 fanus Information about the technological characteristics is, after all, essential for making the records readable. Furthermore a description of the original technological environment could also be interesting for the researcher who wants to know the nature of the organisation at a point in time, especially when the records are transferred to the archives or when the organisation no longer exists. The big difference with the paper world is that nothing seems to be obvious any more. Familiar characteristics of authenticity as the signature, the seal, the paper form which fixes the information of the record, are disappeared or have changed. Archivists and record managers however are familiar with the formal characteristics of paper records, which make them authentic, reliable etc. As a result they could be able to identify them so that they can be implemented in the electronic enviroment. Because this is in the case of records a virtual environment it has to be done explicitly otherwise information will be lost. One aspect should be clear, however: the description itself will mostly be electronic as well, and will need to be managed too. As a result it seems there is a kind of permanent recurrence in this virtual world of electronic records. Not only the records need to be described, the description and the management of the contextual information as well in order to enable control and access. ARCHIVAL DESCRIPTION OF ELECTRONIC RECORDS Having established, in general terms the physical aspects of electronic records and the nature and role of intellectual control and having examined what is or should be done in this field within the record creating organisation (as part of the record keeping function), it is useful to focuson an activity of particular importance: archival description. In discussing archival description we need to be aware that the viewpoint has changed. We now take the perspective of the archives, to which those records of archival value are transferred. The main activities of the archival institution are the preservation of archival records through time and maintaining them in the same condition as when they were received. An essential activity is to make them accessible and available for research. This is accomplished through archival description. Furthermore it is no longer the mdividual record that we are focusing on, but the fonds or group of records which is transferred to the archives. These records are the result of a process of appraisal and selection and as such represent evidence of what a government organisation has done in accomplishing a function. The purpose of the activity of archival description is to make and keep records accessible and to take care that they are (and stay) understandable, so that the researcher can find the information he is searching for. Within the archival environment the notion of authenticity of the archival records also plays an important role. The UBC project contends that it is the responsibility of the archives to maintain the authenticity of archival records, and that it is the best guarantor for the authenticity of the record. The starting point for maintaining authenticity is the point at which they are received by the archives. At the moment of receipt the archives consider the records as

7 Janus Î998.Î Lligall 12 Barcelona authentic archival records. It is not the task of the archives to judge whether or not the records represent what has happened within the record creating organisation. What impact has the phenomenon of electronic records had on archival description? Does anything need to change? When we examine the fact that the record is hardly a physical entity one would expect that something will change. It is not the message, it is the medium and the context so to say, that are relevant for the archivist. This is the information he provides for himself in order to manage them and for researchers to enable them to understand and interpret the message. With paper records the medium, the content and most of the contextual information form, physically, an entity. When they are transferred to the archives, however it is necessary to add some contextual information, because they are leaving their original record creating environment, as well as the original record keeping system. This institutional information tells something about the record creating organisation and about the record keeping system (the rules) including the relationships among those records having archival value. This information about provenance and structure is even more crucial for archival records. We need to be aware that this is contextual information on a different level than that normally required for the individual record. We must now operate on the level of the fonds or record group or series. In this situation we can, better than in a paperhased environment, build upon the contextual information about the business process captured in the course of the creation of a record and the contextual information about the records management stored in the electronic record keeping system (during the maintenance stage). The purpose of archival description is to make it possible for a researcher to find the information he is searching for and to interpret the records correctly. As a result it is important to know the expectations of the researcher. First of all the researcher wants to get the information. Secondly he expects the documents which contain the information to be authentic and if possible reliable. Partly this is achieved by the 'unbroken custody' of the records by the expert, being the archivist, as Jenkinson called it, on which he can rely. The UBC project refers in this respect to the archives as to a 'secure place', where the archival records are being preserved. Furthermore, in the paper world, the researcher can see the document and judge whether the document is authentic, for instance by feeling the paper or parchment, by looking at the handwriting, the seal or the signature or other characteristics he can assess its authenticity. In the electronic world, as we have seen, there are no physical characteristics of this kind. The researcher must rely on what the archivist says (i.e. there is no original document). What he sees on the screen is a representation of what is stored on a disk as zeros and ones, made visible and readable only through the use of software. By describing the elements of the original environment (as previously mentioned) and of the structure of the fonds the researcher obtains an idea of the status of the record, the role it has played within the business process and the transaction for which it provides evidence. Both groups of elements should be included in the record keeping system, which can be defined roughly as a system that safeguards the authenticity and reliability of electronic records. As a result, the record keeping system as a whole should be described, because this system controlled the records while they were still within the record creating organisa-

8 134 Barcelona Lligall 12 fanus tion. It contains the rules such as the definition of a record, informtion on the capture of records, appraisal, retention or disposal schedules, and the arrangement of the records. Given that few real electronic records keeping systems exist, we have to rely on paper documentation for the information, and in the case of electronic records, hope that this documentation will be updated when the record keeping rules change. The top-down or retrospective approach to archival description is contained in the General International standard of archival description (ISAD/G), which describes the (26) elements of description. The ISAD/G distinguishes different areas of description, such as the identity statement area, the context area and the content area. Part of the last area is related to the structure or the arrangement of records.' Contrary to the past, the physical arrangement of records is, in the case of electronic records, no longer an issue. In fact, in an information system, physical location is determined by the operating system. Physical arrangement has become virtual which should not be very strange because physical records no longer exist. That does not mean that we do not have to describe their logical arrangement, or their structure. On the contrary... The logical arrangement is determined by the business process and the procedures used to fulfil this process. As such it is part of the structure that should be described. An example of such a logical arrangement is the classification scheme. Technological aspects/change There is in the case of electronic records also the problem of technological obsolescence. It is well known that technology changes quickly. The original software may not be available for electronic records after 5 or 10 years because the hardware platform does not exist any more or the storage format has changed and can not be read by the software. It has already been mentioned that the first thing that will change is the physical aspect. The rapid obsolescence of hardware and software forces archivists to migrate and convert electronic records tokeep up with new generations of hardware and software. Migration and conversion is possible, but only with the necessary guarantees of authenticity and reliablity. The question is how to migrate all or part of the original functionality of the information system or record keeping system through time? Although there are several ways of preserving the original software, there is very little practical experience, especially with modern electronic records which are more complicated then statistical database files. In this respect we can use the documentation about functionality to present electronic records in the way they were once created and used. We can also develop viewers which can present the records in their original form. These viewers obviously don't have the same functionality as the original software. The only function they have is to present the records on the screen in the same way as they were once available. As we have had very little experience with viewers, we can only imagine whether they will be sufficient or not. We also don't know if this approach would be applicable for all kinds of different, sometimes very complex records? Yet we also need to be aware that this option will never represent the original technological environment (or platform). Technology will have changed so much that,

9 Janus Î998.Î Lligall 12 Barcelona for instance, the speed of the new system will never be the same as the speed of the original soft- and hardware. An additional and interesting aspect of using viewers is that one could include new ways of searching, a situation that could be of help to researchers. In some respects, this can be seen as a form of description, Just as we do in a paper environment when we add indexes. Whether the development and use of viewers will be the most cost-effective approach, I don't know. There are other ways to accomplish this such as preserving the original software and data as objects. The only thing I can say now is that we will have to experiment and find out what are the critical factors in this respect. It is clear that archival description plays an important role in safeguarding the authenticity of electronic records both on a technological and institutional level. If the documentation or contextual information accompanying the transferred records is defective/poor, the archives can try to redress this and make it more complete (i.e. through research and description), or the archival description will be poor. Even adequate archival description the story is not finished, because we have to deliver and present the information from our descriptions to the researcher. The new information technology offers us new opportunities and possibilities to do that, but that in fact is a different game, which is outside of the scope of this paper. At this point I only want to indicate some of the possibilities. We should be aware of the fact that electronic records consist of different parts which also can be addressed directly, just because they are electronic. That can be an advantage for making them more accessible. For instance is full text retrieval of the content possible. It can be useful, but the question of whether this will be always the most efficient or adequate way of searching/retrieval is still open. Both the contextual information recorded in the creation stage and the information based on archival description can be used in a retrieval system which should guide the researcher to the information (and the records) he is searching for. It is necessary, however, to design the user interface by which this information is presented to him or her in a way that is as user-friendly as possible. This means for instance that it is oriented in a way that is most sensitive to the way in which researchers ask questions. This user interface or access system should help users in formulating their question more precisely and lead them step by step through the different layers of contextual information toward the archival records they are looking for. Keywords, geographical or personal names or the organisational structure are but a few of the examples that can be used. Also full text retrieval mentioned already should be possible. The challenge is to design an user interface which will enable all these things and at the same time will be independent of the original technical infrastructure. SUMMARY AND CONCLUSIONS It was my intention to make some observations about the intellectual control of electronic records. I hope I did'not make it more confusing for you. Therefore I will try to summarize what I have said.

10 134 Barcelona Lligall 12 fanus The general conclusion is that intellectual control has to play a more prominent role in the information age, because electronic records are not physical entities. That does not sound very shocking, but I think it will be very fundamental for the strategies records managers and archivists use to define records and record keeping systems in the near future. In this respect we can distinguish two viewpoints. The first reaches (?) from the beginning of the life cycle of records, the creation (or perhaps even the conception) stage, which represents in fact an incremental way of describing the records and their environment. The other is the retrospective view of archival description, which completes the information about the transferred archival records and enables future researchers to get an idea how it was when the record creating organisation still existed. It places or should place the records in their right historical perspective. Both perspectives are necessary for ensuring authenticity, understandability and accessibility of records in general and of electronic records in particular. Does that mean that archival description in an electronic environment will change fundamentally? I don't think so. The purpose of archival description, that is to make records accessible and understandable for the researcher, is still the same. The underlying technology however has changed and to continue to change rapidly. The electronic record is no longer a physical entity. Data and rules (or structure) are separated. Software is needed to present the record on the screen. Technological obsolescence requires archives to use special measures to ensure the availability, readability and understandability of records through time. But most important, the original environment in which the eletronic record was created and used, has to be described more explicitly than in the case of paper records. Only then is it possible to give the researcher an impression of the authenticity of the electronic record. This authenticity is based on both description and the idea of 'unbroken custody'. In this respect description needs special attention, because the 'look-and-feel' aspect of the record does not exist in the digital world. Another approach is required. The basis of adequate description lies however in the beginning of the life cycle of records. There, description starts with the recording of contextual information. The identification and capture of electronic records, including then contextual information into a record keeping system is necessary because it is needed for authenticity purposes. As a result, it should control both paper and electronic records. In conclusion, I would want to leave you with the statement that description will be the most stable factor in the virtual world of electronic records. The physical components will change continuously because of techological developments. We have to be aware however that the description itself will mostly also be in a digital form. So how to deal with this recurrent character of electronic records? This conclusion leads to some interesting philosophical thoughts. Walking through Barcelona with a colleague we visited the still unfinished cathedral 'La sagrada Familia'. In discussing electronic records we saw a parallel between the attempt of Gaudi to materialize an untouchable thing as his belief in God into a cathedral and in our attempts to catch elusive electronic records. Human beings have a strong tendency to leave traces behind them of what they think or did. The paradox with electronic records is that we want to record information, but unless we do something we won't leave any trace of our activities. That is in fact the reverse direction or the world upside down.

11 Janus Î998.Î Lligall 12 Barcelona Otherwise one could even say that just because of the dynamic and virtual world of digital information it becomes even more important to capture,. preserve and maintain relevant records. So in this respect the function of records manager or archivist will become more important and that is a reassuring thought. NOTES 1 See for instance, 'Preserving the Present', The Hague 1993, p... 2 I use description in this context as the activity of adding contextual information to a record or to put it otherwise, adding contextual information to a record is a way of describing. In a record creating organisation this is done as mentioned because of reasons of supporting the business process and accoxmtability (integrity, authenticity), m an archives because of increasing the accessibility of records and because of enabling the proper interpretation of an individual record or series of records. 3 In general the elements identified by ISAD/G can beused also in an electronic envirormient.