WALIS DATA MANAGEMENT GUIDELINES

WALIS DATA MANAGEMENT GUIDELINES 13 NOVEMBER, 2006 walis-data-management-plan-template-v01-00.doc i

TABLE OF CONTENTS 1 INTRODUCTION... 1 2 SCOPE... 1 3 WALIS AGREED DATA STANDARDS... 1 3.1 DATA MANAGEMENT PLAN... 2 3.2 DATA DICTIONARY... 2 3.3 METADATA... 2 4 DATA MANAGEMENT PRACTICES... 2 4.1 ENSURING EFFECTIVE CONSULTATION WITH USERS... 2 4.2 DETERMINING PRIORITIES FOR DATA CAPTURE... 3 4.3 MANAGEMENT OF DATA ACQUISITION... 3 4.4 COMPLIANCE WITH RELEVANT LEGISLATION... 3 4.5 DATA LICENSING... 3 4.6 DISCLAIMER... 4 4.7 FUNDING... 4 4.8 COPYRIGHT... 4 4.9 MAINTENANCE AND REVISION... 4 4.10 EVALUATION AND MONITORING... 5 4.11 STORAGE AND ARCHIVING... 5 5 ACKNOWLEDGEMENTS / FURTHER INFORMATION... 5 APPENDIX A WALIS DATA MANAGEMENT SUMMARY TEMPLATE... 6 APPENDIX B DATA DICTIONARY... 10 APPENDIX C LEGISLATIVE ISSUES... 11 APPENDIX D WALIS CUSTODIAL DATASETS SELF- EVALUATION SCORECARD... 13 APPENDIX E PUBLISHING WALIS ENDORSED CUSTODIAL DATASETS... 14 walis-data-management-plan-template-v01-00.doc ii

WALIS DATA MANAGEMENT GUIDELINES 1 INTRODUCTION Agencies in Western Australia possess large amounts of spatial data. The collection and maintenance of this data requires a large financial commitment; yet some agencies diminish this investment through poor data management. To maximise this asset, agencies should adopt systematic management processes for the whole data life-cycle (Figure 1). Problem/idea Data archival/disposal Data specification Data maintenance Data creation Data storage Figure 1: Spatial data life-cycle 2 SCOPE This document proposes in high-level terms the standards and practices to be adopted by WALIS agencies in the management of their data. It was written specifically to support the implementation of the WALIS Data Custodianship Policy, which refers to these guidelines in regard to the agreed data standards to be implemented for custodial datasets. While the agreed data standards discussed in this document are seen as binding requirements for custodial datasets, the general practices discussed are promoted as guidelines only, and deliberately avoid being too prescriptive. Agencies are expected to responsibly research and develop data management practices that match to their business needs, and to apply sound data management practices to all datasets and information products under their charge. 3 WALIS AGREED DATA STANDARDS Data management is a term which covers diverse activities. In the context of the WALIS Data Custodianship Policy, custodians are required to manage their datasets in accordance with agreed data standards. As a minimum, this requires for each custodial dataset that an agency manages, it maintains an up-to-date: Data management plan; Data dictionary; and Metadata statement. walis-data-management-plan-template-v01-00.doc Page 1 of 15

Requirements in respect of these three minimal standards are described below. 3.1 DATA MANAGEMENT PLAN The Custodianship Policy requires that custodial agencies maintain a data management plan. A data management plan is a useful tool that will assist agencies to manage datasets over their entire lifecycle in an efficient and coordinated manner. It allows agencies to plan for the future and focuses all people associated with the dataset on the key data management issues. To assist agencies in developing a dataset management plan, Appendix A provides a data management summary template which when completed would meet minimal compliance requirements. Agencies are encouraged to extend the summary and/or look at other methods to create internal data management processes to suit their business needs. 3.2 DATA DICTIONARY For custodians to meet the agreed data standards in the WALIS Data Custodianship Policy they need to maintain an up-to-date data dictionary. The minimum requirement would be having descriptions for all field names and the attribute values in the fields they deem most important to interpret the datasets/information product or key interpretive fields. Appendix B provides an example of what would meet minimal compliance requirements. Again, agencies are encouraged create their own data dictionaries that meet their own business requirements. 3.3 METADATA Metadata needs to be updated regularly in accordance with the appropriate standard outlined in the WALIS Metadata Policy. This is currently the ANZLIC Metadata Guidelines (Version 2), although this will change soon to a new ANZLIC profile that is compliant with the international metadata standard (ISO 19115). Metadata is critical in enabling internal and external users to locate and determine the usefulness of datasets. The custodian is expected to facilitate efficient and effective use of government data, to derive maximum benefit from the investment for the state and for the custodial agency. Thus the metadata must be readily available for both existing users and potential users. The custodian should maintain publicised points of contact for enquiries and be in a position to provide the appropriate metadata promptly. Processes should also be enacted so metadata is updated regularly and efficiently, as many fields can become out-of-date quickly. 4 DATA MANAGEMENT PRACTICES In addition to developing an appropriate data management plan, custodians should also endeavour to implement the following data management practices. 4.1 ENSURING EFFECTIVE CONSULTATION WITH USERS Throughout the life-cycle of any dataset/information product, the custodian has an obligation to notify users of actions which may affect the subsequent use of the data. This includes consultation over: priorities for data capture; timeframes for data capture and associated processing; data specification; quality standards; walis-data-management-plan-template-v01-00.doc Page 2 of 15

development of data standards; transfer formats; appropriate means of dissemination or publication; data certification regarding reliability, quality and fitness for particular uses; and errors found and changes to original data that make derived data out-of-date. 4.2 DETERMINING PRIORITIES FOR DATA CAPTURE The right to determine priorities for data capture generally resides with the data custodian, although this may be over-ridden by government policy imperatives. However, the custodian also has a responsibility to consult with users and potential users of the data, and to balance objectively the needs of external users, with those of the custodial agency. This is particularly important when other agencies come to rely on the dataset(s)/information product(s) to underpin their own business needs. As new requirements are identified by the WA Government, further demands are likely to be made on agency datasets. These demands may identify data improvement requirements that are beyond the current business demands of the custodial agencies. Effective business cases and service level agreements may need to be developed to support these broader whole-ofgovernment needs. 4.3 MANAGEMENT OF DATA ACQUISITION The custodian is responsible for all aspects of data acquisition. This includes the following: definition of the specification for the initial data and any revision requirements (in consultation with users); definition of standards (in consultation with users); quality management and assurance during acquisition; and drawing up, letting and monitoring of acquisition contracts or agreements, if applicable. The custodian should also ensure that the government gets value for money in the acquisition of data. To avoid duplication, prior to capture of any new dataset, the agency should ensure that the data being captured is not already held by another agency in the format required or in a similar format. Improved efficiencies will need to be established to ensure whole-of-government participation in the maintenance of certain WALIS custodial datasets. Advances in technology will likely provide a technical environment to support distributed maintenance, with supporting agreements within the relevant custodial arrangements. 4.4 COMPLIANCE WITH RELEVANT LEGISLATION Agencies are responsible for ensuring that for all activities such as capturing, storing, using and supplying of the data, that compliance with any relevant legislation is maintained. Acts which are relevant include but are not contained to those on Freedom of Information, Copyright and Records. Detailed enquiries should be directed to the State Solicitor s Office or other appropriate source. Appendix C provides additional information on the various legislative requirements. 4.5 DATA LICENSING Having an appropriate data license for your spatial data is important to regulate access and use of your agencies data. The WALIS website has templates for the licensing of data: (http://www.walis.wa.gov.au/policies/licensing_agreements/). walis-data-management-plan-template-v01-00.doc Page 3 of 15

4.6 DISCLAIMER To limit the government s potential liability for the release of incorrect spatial data, a disclaimer should be included in any license agreement. A link to the relevant metadata statement is often used in the disclaimer. Advice on appropriate words should be sought from the State Solicitors Office, or other appropriate legal source. 4.7 FUNDING It is possible that collective funding will be required to ensure that WALIS custodial datasets are maintained to any agreed whole-of-government standards. The State Land Information Capture Program is an example of a collaborative approach to capture. For all other datasets, organisations should endeavour to share the cost of acquisition through collaboration with other organisations. For example, if users require the custodian to collect data to a higher precision to meet a specific standard or other refinement, the custodian should negotiate with the users over meeting the additional cost. The cost of maintenance and revision will normally be borne by the custodian. However, if a user wishes a revision to be done more promptly, or to a new specification, or to extend the scope, then this should be explored between the user(s) and the custodian through WALIS. 4.8 COPYRIGHT If the acquisition is sub-contracted, the custodian must also ensure that the contract states that any copyright is maintained by the State, if appropriate. If the data being acquired is a compilation of existing data over which others have copyright, the ownership of the copyright in the integrated data is a matter for negotiation and agreement between the custodian and the data provider(s). 4.9 MAINTENANCE AND REVISION To preserve consistency and integrity, each dataset should only be updated under the supervision of the custodian. Organisations should have a defined maintenance frequency and a list of triggers that would require maintenance to be undertaken. They should alert users to subsequent modifications to the original data. A report of an error could be a trigger and organisations should have clear feedback processes in place that facilitate the reporting of errors. Once an error has been reported, organisations have an obligation to correct faulty data and should endeavour to notify affected parties. The SLICP (State Land Information Capture Program) was established as a whole of government data development program to maximise the benefit of the State Government s investment in land-related information in relation to particular data themes e.g. imagery, cadastre and topography. Its role also includes advertising its data capture programs and promoting shared revision of datasets to meet broader government, industry and community needs. The SLICP is underpinned by a regular capture cycle, however it allows for project specific data to also be collected. Agencies and organisations need to be aware of the SLICP and its potential to reduce maintenance and revision costs. walis-data-management-plan-template-v01-00.doc Page 4 of 15

4.10 EVALUATION AND MONITORING The WALIS Data Custodianship Policy requires custodians to conduct a self-evaluation of their compliance to the agreed standards. A Self-Evaluation Scorecard has been provided (Appendix D) to assist custodians in performing their self-evaluation. Agencies should also develop processes to monitor their data management processes to determine the use and effectiveness of their datasets. This might include audits of the data collection, maintenance, storage and dissemination activities and their level of compliance with internal data management procedures and policies. Agencies also need to continually evaluate the quality of their datasets by determining whether they are still fit for purpose as defined by the users of the data. Questions that should be asked include: - Are the user s needs being met? (whole of government) - How will maximum use of the data be made? and; - Are the costs of handling, storing and maintaining these data acceptable? 4.11 STORAGE AND ARCHIVING The custodian of the data should comply with the State Records Act 2000. (http://www.sro.wa.gov.au/about/sra2000.html). Under the Custodianship Policy, custodians are required to ensure appropriate storage, maintenance, security and archiving of their custodial datasets.. This means the provision for suitably secure copies of the dataset, while allowing appropriate access to the data. In addition to storage, there needs to be an appropriate archival procedure that includes a migration strategy to transfer data to new technology. Custodians need to be cognisant of the value of historical data to the wider community, and should take a responsible approach in any considerations to reduce or curtail access to data through migration and archiving of historical data. A broad range of stakeholders need to be involved in these processes. 5 ACKNOWLEDGEMENTS / FURTHER INFORMATION This document draws on material produced by the: - British Geological Survey, Internal Report IR/00/09R: Guidelines for the preparation of Data Management Plans. - IGGI (Intra-governmental Group on Geographic Information): The Principles of Good Data Management. (http://www.iggi.gov.uk/publications/index.htm) - OSDM (Office of Spatial Data Management) Australian Government Custodianship Guidelines (http://www.osdm.gov.au/osdm/custodianship.html) walis-data-management-plan-template-v01-00.doc Page 5 of 15

APPENDIX A WALIS DATA MANAGEMENT SUMMARY TEMPLATE Below is a Data Management Summary Template which can be used as a template for a Data Management Plan. If custodians complete the summary, this would meet minimal custodianship compliance requirements. Custodians are encouraged to develop their own data management plan that incorporates all subject matter from the summary and any other information that would meet their business needs. Name Name of Dataset/Information Product: Other Names for Dataset/Information Product: Key Responsibilities Custodians of dataset/information product (branch/division): Point of contact (for enquiries incl. distribution): Description Abstract: Provide a brief description of the content, purpose and lifespan of the dataset/information product. Lineage: Summarise the process steps used to design and construct the dataset/information product. Please also list the name, custodian, date, scale and presentation form (map, dataset etc) of any data from which this dataset has been derived. Consultation with other providers/users: Who were consulted in the creation of this dataset/information product? Exclusions: Was anything excluded from the dataset/information product? (e.g. streams without surface flow) Legislative requirements: List any Acts or Regulations that require the dataset, or under which the dataset/information product is created and maintained. walis-data-management-plan-template-v01-00.doc Page 6 of 15

Data Maintenance and Storage Maintenance frequency: Indicate the current maintenance frequency, list any known maintenance triggers Standards and policies: List relevant internal/external standards and policies that apply to this dataset/information product. Quality assurance: Describe the processes and measures undertaken to ensure the data is as correct as possible? Feedback/error reporting: What are the processes to accept feedback from users? Storage: Describe how the data will be stored (e.g. format, location). Archival process What is the process for archival of this dataset/information product? Metadata: Where is the metadata located and what workable plan do you have to collect/maintain it (e.g. metadata Monday)? walis-data-management-plan-template-v01-00.doc Page 7 of 15

Data Attributes/Fields (Data Dictionary) List and describe all the fields of the dataset/information product: Data Distribution Specific distribution instructions: How will it be distributed e.g. WALIS clearinghouse, SLIP, DVD, FTP, internal use only? Usage recommendations: For example, are there any recognised inappropriate uses? (e.g. scale abuse) Access constraints: Are there any restrictions on who may use the data? Proposed Amendments Are there any proposed improvements to the datasets/information products? If so: - Describe level of involvement of stakeholders in assessing amendments - What are the improvements and the reason(s) for it? - When? (e.g. schedule/timeframe) - What are the funding implications? walis-data-management-plan-template-v01-00.doc Page 8 of 15

Other Information Have you considered these issues with regard to this dataset/information product (please tick): Privacy issues FOI issues Copyright State Records Act 2000 (Archiving) Prepared by: Checked by: Approved by: Date: Date: Date: Recommended Review date: walis-data-management-plan-template-v01-00.doc Page 9 of 15

APPENDIX B DATA DICTIONARY To meet minimal custodianship compliance requirements, a data dictionary needs to contain descriptions of ALL field names and the attribute values in the fields they deem most important to interpret the datasets/information product or key interpretive fields. As custodians know their data best, they must decide which fields are the key interpretive fields. Below is an example of a data dictionary that illustrates the minimum requirements. DATA DICTIONARY Descriptor Table for dataset Field Field Field Name Description Type Width UFI N Unique Field Identifier CREAT_DATE D The date the polygon was created. STATUS C The status of the activity Etc * Please note that for Field Type, N = numeric, D = date, C = character Attribute Values Table for STATUS field Values Description C Activity completed P Activity in progress N Activity not complete (Provide a separate Attribute Values Table for each key interpretative field) Attribute table from generic dataset used in the above data dictionary UFI CREAT_DATE STATUS 1 08-05-2005 P 2 01-12-2004 C 3 23-10-2003 P 4 01-12-2003 N 5 27-06-2003 P 6 17-07-2003 C 7 30-07-2004 C 8 23-04-2004 C 9 08-05-2004 C 10 28-11-2004 C 11 18-11-2005 P walis-data-management-plan-template-v01-00.doc Page 10 of 15

APPENDIX C LEGISLATIVE ISSUES The following information is included to provide an overview of some of the legislation that data custodians may need to consider in managing their datasets. This is provided as a guide only and is by no means complete. Readers are expected to investigate likely requirements more fully. Freedom of Information Under the Freedom of Information Act (1982), the public may request access to government documents and, unless the document is subject to a specific exemption in the Act, the government agency must provide that access. 'Document' in the FOI Act includes maps, plans and 'any article on which information has been stored or recorded, either mechanically or electronically'. A special section of the Act refers to digital data. A document that is open to public access and available for purchase in accordance with arrangements made by an agency; is not subject to the FOI Act. For this reason, the FOI Act can not be used to obtain data available commercially. For more information: Freedom of Information website http://www.foi.wa.gov.au/ Privacy At present, no state privacy legislation exists however it is prudent to adhere to the principles of the Privacy Act 1988 (Commonwealth) until we have our own legislation as it is best practice. The Commonwealth legislation applies not just to data with individual names attached, but also to data to which a person s identity can be reasonably ascribed. Thus the cross-linking of datasets in geographic information systems creates a danger of contravening the legislation. Section 14 of the Privacy Act 1988 (Commonwealth) lays down a series of 11 Information Privacy Principles (IPP s) for dealing with personal information. A breach of these Principles is an interference with privacy. Under the legislation, the collector of personal information must inform the individuals of what the data will be used for when it is collected. Records of personal information should be accurate, relevant, up-to-date, complete and not misleading. Data collectors, custodians and users must not use or disclose personal information for any purpose other than that for which it was collected and provided. Data custodians are responsible for ensuring that users are aware of the implications of the Privacy Act and the restrictions on the use of personal information. Currently the mechanism for this is the licence/memorandum which authorises use of the data, subject to certain conditions, including provisions relating to privacy. The custodian is obliged to provide a level of appropriate security to protect the privacy of an individual's data. Individuals have the right to access data held about them. Thus the custodian must be in a position to supply this if necessary, without undue delay and in a form which is readily intelligible to the individual. Each government department is recognised as a separate body for the purposes of the Privacy Act. The legislation therefore applies to data passed between individual agencies. The Privacy Act 1988 (Commonwealth) does not currently apply to non-government and state agencies. So, there is potential for the breaching of an IPP where data is captured by a private sector contractor or used by non-government individuals or organisations. The custodian must ensure that any such data remains secure, and include provisions for compliance with the Privacy Act in the data collection contract or licence agreement. For more information: walis-data-management-plan-template-v01-00.doc Page 11 of 15

http://www.anzlic.org.au/policies_privacy.html http://www.privacy.gov.au/ Copyright In general terms copyright (discussed further below) is the exclusive right given, to the "author", by the Copyright Act 1968 to reproduce, publish, perform, broadcast and adapt a work. This exclusive right may be assigned or licensed to others. The State is the owner of copyright in works (including spatial datasets) made, or first published, under its direction or control. The Act details the works in which copyright subsists, and describes the exclusive rights of the copyright owner, the most important of these being the right to stop unauthorised reproduction of their work. Within the State proper, which is a single legal entity, particular portfolios and included agencies are said to exercise 'custodianship' of copyright in works produced under their direction or control. By convention, a transfer of data between government agencies is under a non-binding arrangement or memorandum of understanding, incorporating terms similar to those which might be contained in a binding licence agreement. The application of copyright ownership rules for statutory bodies is not straightforward - legal advice should be sought to clarify the position. For more information: http://www.copyright.org.au/ Information Sharing Between Government Agencies The Western Australian Government has endorsed a policy framework (http://www.dpc.wa.gov.au/psmd/pubs/legis/premcirculars/2003_05upd.pdf) to facilitate sharing information on a structured basis, particularly confidential client information. This is an interim arrangement until the development of broader information sharing legislation or privacy legislation is finalised. The policy framework aims to build on the information sharing that is currently occurring between agencies. It recognises that the community is demanding seamless services from agencies, and expects positive outcomes from government spending. Structured sharing of important, and often sensitive, information is frequently needed to achieve improved community outcomes, benefits to clients and better coordinate services. As government agencies have a duty of care to all clients, especially children, information held by other agencies can be crucial to safeguard the welfare and safety of clients. The aims of information sharing between government agencies are, therefore: improve community outcomes; improve client outcomes; and better coordinate services. Consumer Protection Licence to Use agreements drawn up between the custodian and the client will, under certain circumstances, come within the ambit of consumer protection laws. For more information: www.docep.wa.gov.au walis-data-management-plan-template-v01-00.doc Page 12 of 15

APPENDIX D WALIS CUSTODIAL DATASETS SELF- EVALUATION SCORECARD Agency: Scorecard: Dataset /information product name Data management plan complete / current? Metadata statement complete / current? Data dictionary complete / current? Dataset quality acceptable? Dataset 1: Primary Point of contact: Comments: Dataset 2: Primary Point of contact: Comments: Dataset 3, etc Primary Point of contact: Comments: Overall Agency Comments / Qualifications Approved by: Name: Position: (WALIS Councillor [as a minimum], or Executive member) Signature: Date: walis-data-management-plan-template-v01-00.doc Page 13 of 15

APPENDIX E PUBLISHING WALIS ENDORSED CUSTODIAL DATASETS Procedure for publishing WALIS endorsed custodial datasets in the Premier s Circular and WA Government Gazette (FOR WALIS OFFICE USE) a) The initial copy will be published in the Premier s Circular. This will alert all CEO s to the release of the WALIS Data Custodianship Policy and what datasets/information products they are responsible for. The Circular will contain a brief summary of the Custodianship Policy and URL links to the: o Current list of WALIS Endorsed Custodial Datasets; o WALIS Data Custodianship Policy; and o WALIS Data Management Guidelines. PROCESS See document - Administration of Premier s Circular s (http://www.dpc.wa.gov.au/psmd/pubs/legis/premcirculars/admin.pdf) This will be a one-off publication. b) Whenever there is an update to the list it will be published in the Government Gazette and on the WALIS website. There will be a preamble and reference to the previous gazette in each Government Gazette. The following heading and sub-heading will be used. HEADING - SUB-HEADING - WALIS (Western Australia Land Information System) WALIS Endorsed Custodial Datasets The published document would be a table with the following headings: Custodian, Name, Description PROCESS - The EPC, on advice from CMG, approve the name, custodial agency and short description of the WALIS Endorsed Custodial Dataset in question. - The chairman of the EPC sends a letter to the CEO/Director General of the custodial agency inviting them to become a custodian or informing them they are no longer a custodian of a WALIS Endorsed Custodial Dataset. The letter will include a request to approve the custodial arrangements in writing to the EPC chairman. - The custodial agency sends a letter to the EPC chairman approving the custodial arrangements. - The name, custodial agency and short description of the WALIS Endorsed Custodial Dataset in question are sent to the State Law Publisher in soft copy (Microsoft Word ) and hard copy (fax 9321 7536 or mail 10 William St, Perth WA 6000) with a letter authorising the publication of the list (A letter template is attached to this procedure). - The letter can be signed by anyone at WALIS Office with authority to disperse funds and all correspondence should go through Peter Telford (ptelford@dpc.wa.gov.au) 9321 0010 Website: (http://www.slp.wa.gov.au/mapfr.htm) walis-data-management-plan-template-v01-00.doc Page 14 of 15

AUTHORIZATION TO PUBLISH GOVERNMENT GAZETTE (To avoid delays in publishing, all details must be completed) Department:... Branch/Section:... Contact Name:... Telephone No:... Order No:... I hereby authorize the publication of the following notice in the Government Gazette dated next issue and acknowledge funds are available. Authorizing Officer s Signature: walis-data-management-plan-template-v01-00.doc Page 15 of 15