Open Document Standards for the Government Guide

Transcription

1 Open Document Standards for the Government Guide Programma Nederland Open in Verbinding & Forum Standaardisatie Date 13 april 2011

2 Colofon Pagina 2 van 17

3 Contents Colofon...2 Contents Introduction Using standards: when and which Different phases, different standards Different content, different objectives, different standards Decision tree Option: Does PDF/ A-1 offer sufficient functionality? Cautions regarding archiving in PDF Option: Conversion to PDF/A-1a? New and existing documents Wait for PDF/A-2? Implementation considerations Document policy The importance of a migration strategy Emulate or convert? Points for attention during implementation Attention to the user Migrating software Tenders Additional information on implementation of standards Context and further information Policy and standards What are open standards? Standardisation Forum and Standardisation Board The NOiV programme Publishing via the internet The archive regulation The BASIS for sustainability Acknowledgements...17 Pagina 3 van 17

4 Preface We have all become so used to the never-ending stream of digital documents produced by our computers that we rarely take notice of the fact that this may cause irritation and inconvenience to others. For instance, a member of the municipal council attempting to access and read council reports on a netbook while travelling by public transport may find that special software is required which only runs on computers with a specific operating system. Or a blind person who is handed an illegible pile of printouts containing erased text after submitting a request within the framework of the Openness of Government Act. Or a digital leaflet targeted at young people but not accessible from a mobile phone. Or an archivist receiving documents in a fragile, applicationspecific format, etc. Publicly funded bodies cannot use random distinctions between groups of people, people with and without a handicap, the rich and the poor, between customers of IT suppliers A and B or between users of this generation and the next. Within reason, documents must be accessible to all, inside as well as outside our organisations, today and in the future. After all, in the information chains we are working in, the final document we produce as output is the starting point for other people working in that chain. We are therefore fortunate to have open standards, such as ODF and PDF, providing sustainable and accessible methods for storing and sharing information. Such standards are impartial and future-proof and assist government bodies in meeting legal requirements in areas such as accessibility and future-proof archiving. Is this difficult? Not really. This easy-to-use booklet assists organisations and their management in understanding how open standards can be used for documents. It describes how information may be structured and provided in a manner that allows different kinds of users to use it without problems, today and in the future. This booklet also addresses practical questions, as well as use of open standards, in the most suitable and cost-effective manner. Open standards are an important building block of our digital society. As Chairman of the OpenDoc Society, I am pleased that our government actively aims for improvement of quality in this area, amongst other things by means of the action plan Nederland Open in Verbinding (The Netherlands Open in Connection) and the activities of the Standardisation Forum. Is your organisation taking part? Bert Bakker Chairman of OpenDoc Society Pagina 4 van 17

5 1 Introduction The Dutch government has implemented open standards for the semi-public sector. A typical characteristic of open standards is that there are no restrictions regarding the use of the standards by ICT users and providers. Open standards enhance digital information exchange (interoperability). Open standards simplify communication between public and semi-public bodies and government bodies, businesses and citizens. Open standards also increase the independence of software suppliers. The Standardisation Forum and the Standardisation Board were established for the purpose of selecting suitable open standards. Selected standards are placed in two lists of open standards, the best known of which is the Comply or Explain list of open standards. The standards in this list must be implemented. If they are not, an explanation must be provided as to why. Chapter 4 provides further information on open standards and policy. In November 2009, the Standardisation Board placed the document standard PDF 1.7 (Portable Document Format, ISO :2008 Part 1) on the Comply or Explain list of open standards. After ODF (Open Document Format, ISO 26300) and PDF/A-1 (NEN-ISO :2005 EN), PDF1.7 is the third document standard to be included in the list. There are various reasons why the Board has selected these standards, but in all cases the open nature of the standards played an important role. All three standards are open, which means that these standards in contrast to their closed counterparts contribute to reduction of supplier dependence and improvement of interoperability and sustainability. Additionally, each of these standards have their own advantages and disadvantages. The PDF 1.7 and PDF/A standards are now fairly well known and widely used. To a lesser extent, this also applies to ODF (Open Document Format). The purpose of this document is to clarify in which cases these open document standards are best used. It is important to note that this document primarily applies to text-based documents, such as reports, notes, letters, forms and presentations. Such documents may also contain images, multimedia content or interactive elements, but these components always support the text. This document does not aim to provide a complete overview of the standards mentioned, nor does it offer extensive implementation advice or detailed theoretic background information on archiving documents. These subjects are discussed extensively in other documents. We will refer to these documents where possible. This document is primarily intended for I&A coordinators and managers of government bodies who are looking for an introduction to this subject. These persons are responsible for proper implementation and safeguarding of standards in the area of data storage within their organisation. Nevertheless, much of the information in this document is also of interest to a larger audience. Pagina 5 van 17

6 2 Using standards: when and which 2.1 Different phases, different standards Generally, document usage processes consist of five phases. In practice, these phases are not necessarily followed in a fixed order. In the first phase, the document is created. In the second phase, a number of people within the organisation use the document in collaboration. In phase three, documents are exchanged. No further document changes are introduced in this phase. In the fourth phase, the document is published for use by a wider audience. In the final phase, the document is archived. These phases are presented in Figure 1. The suitability of each of the three formats is specified for each phase. Green indicates full suitability. Orange indicates that the format is not always suitable. Red means that the format is unsuitable. ODF PDF/A-1 PDF 1.7 Create Collaborate Exchange Publish Archive Figure 1: Phases and formats for text-based documents As indicated in the above figure, ODF is mainly suitable for documents that allow changes, also referred to as revisable documents. ODF is a file format for information exchange on a daily basis. There are no or few limitations for the recipients with respect to using the document of parts thereof themselves. The final phase for one user may be the starting point for the next user in the information chain. In ODF, the document retains the full functionality during editing, including all metadata, editing history, accessibility characteristics, digital signatures and versions. ODF may be used throughout the life cycle of a document for as long as the document is to remain revisable. ODF can be used if a revisable document needs to be exchanged, published or archived. ODF is not suitable for exchanging, publishing and archiving non-revisable documents. If documents must be frozen, and no further discussion of the content possible (non-revisable documents), PDF 1.7 and A-1 are more suitable. PDF/A-1 is specifically intended for long-term archiving. In simple terms, PDF/A-1 creates a snapshot of the working document, retaining the structure and searchability of the original document. Some data and the ability to easily edit the document may be lost. In some situations, PDF 1.7 will still be required. In the following section we discuss reasons for selecting a particular format. We also discuss the differences between the formats in more detail. Pagina 6 van 17

7 2.2 Different content, different objectives, different standards Decision tree The decision tree below is a simplified representation of the considerations involved in choosing between the standards. It is important to note that the choice for one standard does not exclude the others. The decision tree shows that the document must at a minimum be stored in the format found after completion of the decision tree. A non-revisable document that is published may also be published in other formats, in addition to PDF/A (or 1.7, depending on the functions contained in it). A government body may, for instance, offer an XML version of a document in addition to the original PDF version. The difference between revisable and non-revisable was discussed in the previous section. The format to be selected for revisable documents is ODF. Choosing the format to be used for non-revisable documents is more difficult. Figure 2: Decision tree for open document standards for government bodies Option: Does PDF/ A-1 offer sufficient functionality? The first choice for storing non-revisable text documents is PDF/A-1. This is the preferable format because it is also suitable for long-term archiving. However, it cannot be used in all situations. A text-based document may contain parts or functions that cannot be stored in PDF/A-1. Examples of such documents are: 1. Documents intended for limited revisions by means of annotations. These revisions are added to the document as an additional layer instead of changes to the original content. For documents that can be fully revised, ODF should be used. 2. Publication of feature rich text-based documents. These are text documents with embedded 3D objects, multimedia, GEO information or interactive forms, encrypted documents, etc. PDF 1.7 may be used for such documents. Pagina 7 van 17

8 2.2.3 Cautions regarding archiving in PDF 1.7 We would like to stress that independent documents that are not textbased should not simply be converted to PDF1.7, as this may cause loss of data required for processing. In such cases it may be better to retain and archive the original file or store the document in another sustainable format. A second drawback of archiving in PDF 1.7 is that PDF 1.7 is not, by definition, self-contained, in contrast to PDF/A-1. When a document is saved in PDF/A it can be opened in the future, irrespective of the availability of a particular software environment. If a text document containing, for instance, a video is stored and archived in PDF 1.7, it may not be possible the play the video in the future, if the required software is no longer available Option: Conversion to PDF/A-1a? If PDF/A-1 is selected, a second choice needs to be made between the two versions that are available: A-1a and A-1b. The main difference between the two is that PDF/A-1a retains the logical structure of the source document. This improves searchability of the documents and makes editing text much easier. PDF/A-1b does not retain the logical structure of the source document, affecting searchability. For this reason, PDF/A-1a is the preferred format for archiving. In any event, all government documents originally created as online documents must be stored in PDF/A-1a. Again, some documents cannot be converted to PDF/A-1a, either for technical reasons or because of the cost involved. In such cases, PDF/A- 1b may be used. This applies to: 1. Conversion of analogue (paper) documents and pictures to digital documents. These are scanned documents. 2. Existing / old (legacy) digital documents that cannot be converted correctly to PDF/A-1a because they were created using old software or for other reasons that prevent conversion to PDF/A- 1a. 2.3 New and existing documents The decision tree in the previous section applies to new as well as existing documents, because most organisations already have a digital archive containing text-based documents in a particular file format. Suppose that many of your documents are stored in PDF 1.4. These documents are non-revisable. The first option would be to convert the documents to PDF/A. Should this not be possible, for instance, because the documents contain additional features, further options would be to retain the current 1.4 format or convert the documents to Wait for PDF/A-2? PDF A/2, the format to succeed PDF A/1, is currently under development. PDF/A-2 is expected to contain a subset of the full PDF 1.7 functionality. It is probable that some features now available in PDF 1.7 will not be available in the A-2 version. Nevertheless, all existing PDF/A-1 documents will automatically meet the A-2 standard. At the time of writing, the Pagina 8 van 17

9 release date of A-2 and the functionality were not yet known. The assumption is that A-2 will be released in late in 2010 or early Pagina 9 van 17

10 3 Implementation considerations To anyone who is not yet using open document standards, migration may pose a considerable challenge. Successful migration starts with a clear (new) policy on the use of formats in the context of the organisation s document processes. The next step is migration, if necessary. This requires a proper migration strategy. A final important consideration is intelligent procurement. These three subjects are discussed in further detail below. 3.1 Document policy Standards for exchanging and archiving documents do not exist in isolation. They are part of a well-defined document policy in use within your organisation. Amongst other things, this policy defines when, by whom and where documents are created, edited, stored and managed. It must also be clear where, how and for how long documents must be archived and who must have access to them. The organisation must also decide if all documents are to be stored and managed centrally, in a document management system, or stored locally. The basis for such a policy is building a thorough inventory. This involves establishing the following: Which document flows are used within the organisation. Which applications use documents: Often, far more applications use documents than the Office packages everybody thinks of first. Normally, there are many systems that use documents. For instance, systems such as mid-office applications, business systems, document management systems, content management systems, archive systems, financial software, CRM applications and ERP solutions. It is important to crate an inventory of all such applications and establish for each application whether they support or could support open document standards. Which interfaces are used in the information processes: After performing the above investigation of systems, it is important to establish the relationships that exist between these systems. For instance, where the interfaces are located, and in which interfaces documents are relevant. An inventory should also be made of these. Who the suppliers are and how do we deal with them: Part of the software inventory will consist of packages that do not (yet) support ODF and PDF. What the users need: Migration is not possible if you do not know what functionality your end-users require. Often, only the features of particular applications are compared. But it is advisable to focus on the functionality the users require. Often, this is approximately 20 percent of the functionality offered by the software. Do not forget to involve the know-how of the users. The users are the experts who will be using the documents, and they are able to provide sound input for your document policy. Furthermore, the users are crucial in ensuring the successful implementation of your policy, which is why you should involve them in the process as early as possible. Pagina 10 van 17

11 The following are some questions for evaluation with the users: 1. To what extent do you currently use word processing, spreadsheet and presentation programs? 2. Do you use templates and/or macros? 3. How frequently do you require access to old documents (older than one year)? 4. How frequently do you need to modify older documents? 5. What platforms and operating systems do you use to create and view documents? 6. How frequently do you need to share documents within your organisation and/or with third parties? 7. Do you intend to make your documents increasingly available via the internet? 8. What have you learned from your experience with earlier, similar migrations, for instance from MS Office 2000 to MS Office XP, from WordPerfect to MS Office, from Lotus SmartSuite to Lotus Symphony? The above questions provide an initial basis for establishing a policy. If a suitable policy is in place, it is much easier to create a good mix of different standards being used. 3.2 The importance of a migration strategy Emulate or convert? Emulation or migration (conversion) may be used to safeguard the sustainability of a document. Emulation means that current documents will be retained in their closed formats and an emulation of your current situation will be used. When conversion is selected, closed formats will be converted to open formats. Emulation consists of adjusting the computer environment to create an authentic presentation of a document. An advantage is that the document retains its original form. When conversion is selected, the document will be migrated to an open document format. An advantage of conversion is that the document can be presented to a future user in a manner the user is familiar with. This is also the most sustainable solution, because of the fact that your documents will remain accessible in an open manner. A drawback of conversion is that data may be lost in the migration from original file to target file. It is therefore important to develop your conversion strategy with care Points for attention during implementation A migration strategy may be drawn up based on the difference between the current situation and your policy. This strategy must at a minimum contain the following elements: How to ensure that the document policy is embedded in the organisation: A successful migration focusses on the technology, the organisation and the procedures. Which document formats I need to migrate. Which documents I need to migrate and what should do with existing (digital) archives. Convert? Scan? Which applications must be migrated. Pagina 11 van 17

12 3.2.3 Attention to the user Again, make sure you spend sufficient time preparing the migration together with the users/end-users. It is crucial that users are well informed about the migration process. The experiences of government bodies that performed migrations in the past consistently show that this is critical to success. It is therefore also necessary to adapt your communication to the user. This is not always easy. Every organisation has different types of users. Consider, for instance, policy personnel, desk staff, telephone operators and contractors. Different types of users have different needs, different questions and different skill levels. Take this into account during preparations, training and supervising Migrating software A frequently asked question concerning migration to open document standards is whether this also requires migration to open source software, such as OpenOffice.org. This is not necessary. In most cases you can continue to use your current Office package. It is, of course, possible to combine the migration to open document formats with migration to another package, but this may result in increased complexity. Should you decide to migrate to a new package, it is advisable to do so in a step-bystep manner and migrate to the open standard first. If you are currently using the Microsoft Office package, various methods may be used to realise ODF support. Migration to another (open source software) package is always possible at a later time, if necessary. This is an example of a phased approach. For ODF in the Microsoft Office package, you can use either the support offered by Microsoft (from Office 2007, SP2) or the Sun Microsystems plug-in. Not all solutions offer the same quality and strict support of the standard. At the time of writing, the Sun plug-in offered the best and strictest ODF (1.2) support. Whichever migration method you select, the interconnections with your Office package and other applications, such as business systems, midoffices, financial systems, etc., will always be an important consideration. Make sure you always thoroughly investigate the consequences of migration. 3.3 Tenders The best time to demand open document standards is when calling for tender for a new application or for modification of existing software. Standard texts are available from the NOiV Program Agency that can assist you in handling tenders in an open manner. This is relevant for you because suppliers can assist you with the introduction of a more open architecture for your ICT. Use of open standards will enhance interoperability, allowing linking of ICT systems in an easier and more efficient manner. This will give you the opportunity to improve your service to citizens and businesses. Pagina 12 van 17

13 The idea behind open tenders is that they provide yet another way of actively involving the suppliers in achieving open solutions. Many suppliers will be eager to offer assistance. Refer also to the Open Standards Suppliers Manifest on the NOiV website. The number of suppliers signing the manifest continues to grow. 3.4 Additional information on implementation of standards For additional information on implementing open document standards, please refer to the documentation on open standards in the NOiV wiki at Pagina 13 van 17

14 4 Context and further information This chapter briefly discusses the wider context of using the open document standards. Here, we look at current policy, the nature of open standards and projects affected by the subject of this document. It is not our objective to offer an exhaustive list, but to provide a general indication for anybody wishing to investigate related information. 4.1 Policy and standards In their action plan Nederland Open in Verbinding (The Netherlands Open in Connection), the Ministry of Economic Affairs and the Ministry of Home Affairs and Kingdom Relations have defined the importance of open standards for the Netherlands. The plan states that the use of open standards is crucial for improvement of digital cooperation (interoperability) and supplier independence. Part of the related policy is compiling a list of standards to which the Comply or Explain policy applies. Government bodies must use the standards in this list or clearly explain the reasons why a standard is not used. This policy applies to the public/semi-public sector. In addition to government departments, independent public bodies, provinces and municipalities, this includes education, healthcare en social security institutions. Government bodies are not obliged to set up existing facilities with open standards in the short term. The Comply or Explain principle applies only to the renewal or replacement of ICT products or services with a value exceeding 50,000 euros. For additional information, please refer to the action plan Nederland Open in Verbinding (The Netherlands Open in Connection) and the information on the Comply or Explain principle on the site of the Standardisation Forum. 4.2 What are open standards? A typical characteristic of open standards is that there are no restrictions regarding their use by ICT users and providers. Open standards are the opposite of closed standards, which do have restrictions. A standard is fully open if: 1. The standard has been adopted and will be maintained by a nonprofit organisation, and ongoing development is on the basis of an open decision-making procedure which is open to all stakeholders (consensus or majority decision, etc.). 2. The standard has been published and the standard specification document is freely available or can be acquired at a nominal fee. Anyone must be able to copy, distribute and use the document and make it available for free, or at a nominal fee. 3. The intellectual property rights to i.e. any patents that may exist on the standard, or parts thereof, are made irrevocably available on a royalty-free basis. 4. There are no restrictions with respect to reuse of the standard. Pagina 14 van 17

15 Additional background information on open standards is available from the website of the Standardisation Forum. This is also where you can find the lists of open standards. Additional information on specific standards standard is available in the NOiV Wiki, where information regarding usage is being gathered for the various standards, for instance ODF, PDFA/1 en PDF Standardisation Forum and Standardisation Board The Standardisation Forum is a think tank offering advice for selection of standards and assistance in the development of instruments to promote adoption and implementation of standards. Decisions by the Standardisation Board are based on an advice provided by the Forum. The Standardisation Board publishes and maintains a list of open standards to which the Comply or Explain principle applies. Additionally, the Standardisation Board has drawn up a list of common open standards. Further information is available at The NOiV programme The NOiV Programme Agency (The Netherlands Open in Connection) assists government bodies in the implementation of the policy described in the action plan. The Office offers advice on and promotes the use of open standards. This document is primarily maintained by the NOiV agency. The most recent version of this document can be found in the NOiV Wiki, along with other information on open standards. 4.5 Publishing via the internet If you make documents available to citizens, businesses and other government bodies by publishing them on the internet, you must also adhere to the web guidelines defined by the Dutch government in the web guidelines quality model. The model consists of 125 quality requirements for improved web publications. The objective is to ensure that, for instance, disabled persons or persons with a visual handicap are able to access all information. Use of open document formats fully complies with these guidelines and the accessibility support of ODF and PDF allow for easy adherence to the recommendations of the web guidelines, for instance by providing alternative text for images in documents. You should also ensure that the colour combinations used in your documents provide sufficient contrast, so that visually handicapped persons and persons who are colour-blind are able to read your documents online. For additional information, visit The archive regulation The new archive regulation (an amendment to the Archives Act) took effect on 1 April, This regulation defines the quality requirements applicable to archive regulations and the construction and setting up of archive facilities. The archive regulation acknowledges the relevance of openness for sustainability. It does not mention the formats to be used, but offers a wider perspective on archiving. The full version of the regulations is available at Pagina 15 van 17

16 4.7 The BASIS for sustainability The foundation Digitaal Erfgoed Nederland (Digital Heritage Netherlands) recently published a document called De BASIS. This document outlines a generic sustainability approach for heritage institutions. This approach may be applied in a wider context and provides a useful starting point for actions to be taken by organisations to achieve a complete policy for sustainability. Additional information on the document De Basis is available at Pagina 16 van 17

17 5 Acknowledgements For this document, the following sources have been used: List of open standards for Comply or Explain, Actieplan Nederland Open in Verbinding (The Netherlands Open in Connection Action Plan), ODF handbook, pdf Online session explaining the relationships between ODF & PDF, Introduction to PDF/A, Paper on PDF/A-1, Expert advice PDF/A-1, ISO_Expertadvies_ _2005 PDFA-1_.pdf Expert advice PDF 1.7, 0PDF%201%207%20-%20finale%20versie.pdf Pagina 17 van 17