Data access and management

Transcription

1 B Data access and management CONTENTS B.1 Introduction... B-1 B.2 Data requirements and availability... B-1 B.3 Data access... B-2 B.4 Overall procedures... B-2 B.5 Data tools and management... B-4 Appendix B-1

2 B.1 Introduction This appendix details the dataset requirements for the production of a SMP and should be cross-referenced to Appendix D, which provides details of sources for coastal behaviour and dynamics. A data request procedure is presented and links are provided to the Defra Flood Management website where the full list of relevant datasets is described, together with contact details for the providers where a national provider for SMPs exists. Derived data output and distribution standards for the SMP data and information are provided. B.2 Data requirements and availability The role of a core of nationally available datasets is to provide consistency and compatibility between SMPs developed within regions and nationally. Such core datasets offer efficiencies in acquiring data and standards in managing and using the data; many of them are held by a few organisations and are already available in digital formats suitable for GIS. Certain core datasets are also required within the analysis tools developed for the review of SMPs. Other datasets generated locally or regionally, such as scheme strategy plans and coastal monitoring will not form part of this core data supply but will be used to enhance the understanding and analysis of options within each SMP. The listing of core and associated data is provided in the data table available on the SMP pages of the Defra Flood Management Website ( Defra s contact address is; Corporate Data Licensing Officer Geographic Information Unit Area 5D Ergon House c/o 17 Smith Square, London SW1P 3JR This table will be maintained to reflect updates to the data, availability of additional relevant datasets, and any changes in the supply format. There is a distinction to be made between designations for nature conservation, heritage and landscape (as planning constraints) and datasets describing the environmental interests at a location. Designations alone are inadequate to allow operating authorities to take full account of the biodiversity issues within the SMP. Datasets describing both aspects are needed to develop a responsive SMP; thus the specific habitat and biodiversity data at a site as well as the designation boundary are relevant to the policy appraisal. A number of datasets are still inadequate for evaluation of the defence options or are not available at a strategic level (e.g. data on archaeological potential and value is still scarce and inconsistent). Base topographic data provide both data in their own right and basemaps against which other information is displayed. Various scales of Ordnance Survey digital map data are Appendix B-1

3 recommended to provide the appropriate resolutions to display both the detail and the strategic overview at cell level. Scales of raster data at 1:10,000, 1:50,000 and the OS Strategi data at 1:250,000 provide appropriate scales for these purposes. No specific map output scales have been defined within the procedural guidance (see Appendix I). The table identifies those data that form part of the mandatory supply to the Modelling and Decision Support Framework (MDSF) and those optional datasets B.3 Data access The core datasets are proposed for supply from a limited number of agencies with defined supply routes. The supply of core data will include metadata relevant to the data sources where this is available. The supply routes and processes for data acquisition are defined, against the relevant datasets, on the Defra website as identified above. It is acknowledged that acquiring datasets can take a long time and sufficient lead times are very important for some sectors contributing data to the SMP process. Access to core datasets will help to rationalise this process. The licensing arrangements usually require that the data be used solely for the purposes of creating the Shoreline Management Plan. The metadata supplied with datasets should contain details of the access and use constraints and licensing details. B.4 Overall procedures The following list provides an overview of the procedural steps recommended for data management and output within the development of the SMP. Establish metadata index to record datasets acquired and used within the SMP development process (theme and coverage). Collate all existing datasets generated by the first generation of SMPs and analyse data update requirements (integrate data into GIS and assess the value of the information and gaps). Collate and analyse new information available since the last SMP in particular national studies, results of shoreline monitoring and coastal research data. This will include the access to GIS datasets developed for other planning purposes, such as CHaMPS, Estuary Management Plans, etc. Identify and collate all scheme and strategy plan information within the SMP area. Acquire national standard data package from national agencies and authorities using current guidance from Defra website. Appendix B-2

4 Incorporate all spatial data within GIS system and where appropriate within MDSF. Identify the information gaps and areas where updates or data capture is required. Identify regional and local datasets and acquire and convert to appropriate formats. Integrate data themes to generate thematic maps of the influences on coastal defence policy and to support analysis of options (overview, processes, defence, conservation, heritage and human environment, planning and management). Generate map (GIS) data in portable projects and document formats. Create metadata for all datasets generated within the SMP production process. Create archive copy of datasets for distribution to Coastal Groups. An overview of the data access and management process for the revised SMP is presented in Figure B1, and should be read in the context of the SMP development procedures. Appendix B-3

5 Figure B1. The data access and management process DATA ACCESS & COLLATION Collate and evaluate information from 1 st generation SMPs Request Standard Data Package/s from key suppliers and incorporate in GIS (via proforma) Acquire nationally supplied Modelling and Decision Support Framework (MDSF) package and populate with data Collate information from coastal monitoring / plans, FutureCoast, CHaMPs Estuary Plans (EMPs) Collate regional and local new data relevant to the SMP Collate relevant information from Scheme Strategy Plans (SSPs) DATA MANAGEMENT / ANALYSIS Re-establish and develop GIS / database data management for geospatial data through life of SMP process Identify data gaps and update requirements Identify new spatial data capture requirements Capture geospatial data to specified standard and formats Develop scenarios within MDSF to evaluate policy options Capture data from SMP process (SMP boundary, process and management unit s etc) MAP and METADATA OUTPUTS Establish and maintain SMP databases: bibliographic database consultation database metadatabase Establish and maintain metadata records of all coastal process and shoreline geospatial data GIGateway Generate thematic outputs and maps to support 2 nd generation SMPs presentation / web delivery Create distributable Plan information base and GIS files in agreed formats: GIS / database /portable document formats B.5 Data tools and management A number of good practice guidance notes have been produced on coastal data management (e.g. CIRIA, 2000; MAFF, 1994), which provide advice on handling spatial data in the coastal zone. It is anticipated that GIS will be used in developing the SMP outputs and that the resulting geographic elements of the plan will be available in GIS formats. A key data management objective for the second generation SMPs is to ensure that the data generated within the process are accessible to the Coastal Group and that the data about the data (metadata) are effectively recorded so that the provenance and the limitations of the information sources are understood. Data management within the context of the SMP is considered to include the storage, use and reuse of data, maintenance and archiving of the resources. It is also taken to include the data inventory and transfer requirements. Data management within the scope of the SMP implies a number of elements: Appendix B-4

6 Data and analysis tools; Data inventory / Metadata (records about the data); Technical data management (formats, storage, archive etc); Data licensing (copyright, reproduction, publication and distribution rights); Data creation standards (where new data is created or existing data captured in digital form); Data output, archiving and distribution. B.5.1 Data tools The coastal extension to the MDSF application (Appendix C) provides a basis for collating within a GIS framework much of the spatial data and metadata for the SMP. Although this application is based on a specific GIS product the SMP development need not be seen as using MDSF exclusively. MDSF has a particular functional role within the SMP in providing economic and socio-economic impacts scenarios of varied response options. MDSF provides the basis for defining data standards for integration within a common data analysis tool. This includes the integration of data on coastal evolution, flood risk data and other sources required in modelling. No standard GIS data model has been created for SMP development although this is implicit in the import and format of those data imported into the MDSF GIS data layers. A degree of data cleaning and reformatting is also necessary within the MDSF implementation (e.g. development of a full river network). B.5.2 Metadata Good data management practice recommends mandatory compliant metadata (concise information about a data resource) for all datasets used or created within the SMP process. No national coastal metadatabase product is available to support SMPs, although a number of coastal metadata programmes have been established. The GIGateway data directory from the Association of Geographic Information (AGI) provides a basis for recording SMP geospatial data and an application for recording metadata. Full details of this system and access to the metadata tool are at: GIGateway s metadata tool is designed to assist others in contributing metadata to existing gateway nodes or through a new node dedicated to shoreline data ( A NGDF compliant metadata tool (MS Access database) is available for download. It is intended that the SMP process follows this system and that a metadata node will be created through the AGI to support the SMP process. Management of the metadata should be a centrally co-ordinated activity. This system is compliant with the e- Government Interoperability Framework (e-gif) initiative and the e-government Metadata Standard (e-gms) standard mandatory for public sector bodies. The MDSF application holds metadata records and data quality assessments that contribute to the effective use of the data. MDSF can import metadata and generates metadata records for the data generated during scenario testing. Key data providers to the SMP process have developed their own metadata systems and many of the national core data have metadata records. Appendix B-5

7 The GIGateway is government-funded and the metadata entry tool, which is freely distributed, stores metadata compliant with international standards. The basic identifier elements of the metadata standard are provided in Table B2. Table B2 GIGateway Discovery Metadata elements Identifier Element Name 1 Title 2 Alternative Title 3 Originator 4 Abstract 5 Data Capture Period capture 6 Status of Start Date of Capture 7 Start Date of Capture 8 Status of End Date of Capture 9 End Date of Capture 10 Frequency of Update 11 Presentation Type 12 Access Constraint 13 Use Constraint 14 Keywords 15 Geographic Extent 16 Spatial Referencing by Coordinates 17 System of Spatial Referencing by Coordinates 18 Bounding Rectangle 19 West Bounding Coordinate 20 East Bounding Coordinate 21 North Bounding Coordinate 22 South Bounding Coordinate 23 Spatial Referencing by Geographic Identifiers 24 National Extent 25 Administrative Area Extent 26 Postcode District Extent 27 Spatial Reference System 28 Level of Spatial Detail 29 Supply Media 30 Data Format 31 Additional Information Source Appendix B-6

8 Identifier Element Name 32 Dataset Association 33 Supplier 34 Contact Name or Title 35 Full postal address of supplier 36 Postcode of Supplier postcode 37 Telephone number of supplier 38 Facsimile number of supplier 39 address of supplier 40 Web address of supplier 41 Date of update of metadata update 42 Sample B.5.3 Formats and storage Datasets required within the development of SMPs include a range of database, spreadsheet, geospatial, graphic and image data. No specific technical specification is provided for the development of the SMP, although tools (MDSF, GIGateway) developed for the processing and metadata will introduce consistency. Data management within the scope of MDSF requires datasets in specific formats (Arc shp files), and access to specific software (ESRI, ArcView 3.2a, Spatial Analyst extension and 3D Analyst Extension). The extension to the MDSF application to include coastal and estuarine flooding introduces new data layers but the system input and outputs are essentially the same formats, and the output files for risk scenarios can be exported from the application. For geospatial data generated by the SMP process (Policy Units) or digitised from secondary sources where no appropriate digital data exists (land allocations, etc) a data capture standard should apply. The Environment Agency data capture standard is recommended. The standard should reflect the strategic nature of the SMP process: should identify a minimum scale at which the information would be captured. It is recommended that this be 1:10,000 scale as this is the largest scale at which first generation SMP data was presented; all data should be topologically correct and not contain digitising errors; all spatial data should be attributed. B.5.4 Licensing Licensing arrangements usually prevent the outputs of SMPs maintaining the base data with the spatial data layers that make up the mapping outputs of the SMP. Integration of the existing SMP and Strategy data collections will need to re-establish the existing information base. Appendix B-7

9 Many of the data acquired for existing SMPs were collected from organisations with either express or implied limitations of use to the specific programme, and in some cases limited licence arrangements. This limits the subsequent use of the same data for SMPs revision and also limits the distribution of the resulting SMP data in digital formats. These issues need to be addressed with data providers prior to the development of the SMP revision. Data provided by the Local Authorities and others for the purposes of existing SMPs was usually supplied under licence to the organisation, usually accompanied by a right to supply to the contractor for internal business purposes. Where a number of organisations are involved within a project the data might be licensed to the Coastal Group. In the case of access to OS licensed data the individual LPAs hold separate data licences for the OS data covering their frontage and hence a series of licences are needed for use, reproduction and publication based on these licences. This issue promotes the access to information from a single source where possible. Licensing of the datasets should acknowledge the specific uses within the SMP, the reproduction, publication and subsequent distribution of the resulting information and potentially the development of web based products. Modification to standard licence arrangements may be required to provide for the type of use envisaged by the SMP programme. The licence terms for the transfer of the information and sub-licence may limit the intended approach to distribute the collated information from the SMP process to the members of the coastal group and may limit the use on any internet based SMP. Other download options for data (e.g. from MAGIC) will also need to clear any copyright and licence issues prior to download. Copies of all data licences should be maintained by the consultant and copied to the lead authority. B.5.5 Archiving the data Integrating the existing SMPs and Strategies will be limited by the lack of appropriate archiving of the data; requiring update to the baseline information. In many cases there have been new coastal and shoreline initiatives that have built GIS datasets ignoring the datasets that SMPs have collated. Data management for SMPs should include the archiving of data sources and the preservation of access to the data (through translation if necessary). Guidance on maintaining digital records is provided by Public Record Office (National Archives) Guidelines for Management, Appraisal and Preservation of Electronic Records ( The SMP data distribution is to all Coastal Group members and although some authorities may require data translations to compatible formats with their host systems a standard copy should be maintained of the archive files (including report, graphics, GIS layers and data). Archiving of some of the datasets may not be permitted by the specific licence agreements. In these circumstances metadata are vital if the materials need to be regenerated for subsequent use. Where data reproduction is not permitted it may be necessary to maintain digital copies of the printed diagrams and maps that accompany the SMP rather than the Appendix B-8

10 project files used to create these diagrams. Portable Document Format (pdf) data management is recommended for printed map files. It is recommended that a national archive of SMPs in digital format be established. The development of SMPs accessible online would also help with the wide distribution of the information. Appendix B-9