GInfo. Data Warehouse. Metadata Preparation. Sammy Cheung S&T Division. Portal to Geotechnical Engineering Office geospatial data

Transcription

1 GInfo Portal to Geotechnical Engineering Office geospatial data Data Warehouse Metadata Preparation Sammy Cheung S&T Division

2 Outline of Presentation Objectives of geospatial metadata Standard for preparing metadata Core geospatial data Metadata tools

3 GEO Strategic Goal Development of geotechnical information infrastructure Authoritative source of geospatial data Data Warehouse

4 Management of Geospatial Data Goal 6 Strategic Team reviews the present data available in GEO Identify more than 50 core spatial data that are commonly used by GEO Divisions Images Slope and Catchment Maps Digital Terrain Models Landslide Data Geotechnical Data Project Records

5 Looking for Data? What is the quality of the data? What does it contain? Are they up to date? Where are the data? Does it fit my purpose? I did it years ago and I forgot what's in it? I don't know the meaning of these abbreviations?

6 Metadata Data about Data Describe the content, quality, condition, and other characteristics of data Help person to locate and understand data

7 Why Metadata? Organize and maintain an organisation's investment in data Provide information to data catalogue and data warehouse Provide information to aid data transfer and exchange

8 WBTC 1/96 Documentation for Digital Geographic Data Complete with a set of metadata documentation prepared as per ASTM Section D , Content of Digital Geospatial Metadata Review in every six months Catalogue of Geographical Information System ASTM Standard on metadata has been replaced by Federal Geographic Data Committee (FGDC)

9 Standard for Metadata US Federal Geographic Data Committee (FGDC) FGDC-STD : Content Standard for Digital Geospatial Metadata (CSDGM) Guidelines for development of profiles and user-defined metadata entities and elements

10 Advantages of using Standard for Metadata Supports common use of metadata Provide a common set of terminology, definitions, and information about values to be provided Identify mandatory and options data elements

11 FGDC Content Standard for Geospatial MetaData It is not easy Can be a complex structure with 334 elements Descriptions need to be brief but concise Some elements require good understanding of the data and assessment, e.g. currentness and accuracy of data But Not every data elements are mandatory Allow subjective assessment with qualifications

12 Main Sections in FGDC CSDGM Identification Data Quality Spatial Data Organization Spatial Reference Entity and Attributes Distribution Metadata Reference Mandatory Mandatory Structured syntax, subsections under each section Some elements are mandatory, mandatory if applicable, optional

13 Identification Citation Description Keywords Spatial Domain Access Constraint Use Constraint Time period of contents Status Contact Information Native Data Set Environment Originator Publication Date Title Issue identification Publication Place Publisher Organisation responsible for the data e.g. GEO Date of publishing the data Simple, but concise. Prefer to cover the geographic contents of the Data, e.g. in Hong Kong CGE/S&T, GEO, CEDD

14 Identification Citation Description Keywords Spatial Domain Access Constraint Use Constraint Time period of contents Status Contact Information Native Data Set Environment Abstract Purpose What is the Data? describe the aspects of the Data general content and features data set form (GIS, CAD, image, Dbase) geographic coverage time period of content (begin and end date or single date) special data characteristics or limitations Why we need to create the Data?

15 Identification Citation Description Keywords Spatial Domain Access Constraint Use Constraint Time period of contents Status Contact Information Native Data Set Environment Theme Subject Place Temporal Temporal keyword thesaurus Use as an index to the contents of the Data and can contain: subject : [landslide incidents] place: [Hong Kong] stratum: [Surface] temporal: [post GEO, postwar] None

16 Identification Bounding Coordinates Citation Description Keywords Spatial Domain Access Constraint Use Constraint Time period of contents Status Contact Information East, South, West, North Bounding Coordinates Coordinates in latitude and longitude West_Bounding_Coordinate: East_Bounding_Coordinate: North_Bounding_Coordinate: South_Bounding_Coordinate: Native Data Set Environment

17 Identification Citation Description Keywords Spatial Domain Access Constraint Use Constraint Time period of contents Status Contact Information Native Data Set Environment Describe the restriction for accessing the Data, e.g. "The digital data is copyrighted by the Government of HKSAR." Describe the restriction on the use of the Data after access right is granted. e.g. digital data shall be used in government projects.

18 Identification Citation Information about how "up-to-date" is the Data Description Keywords Spatial Domain Access Constraint Use Constraint Time period of contents Status Contact Information Native Data Set Environment Currentness Reference Progress Maintenance and update frequency Domain: Complete In work Planned Domain: Continually, Daily, Weekly, Monthly, Annually, Unknown, As needed, Irregular and None planned

19 Identification Citation Description Keywords Spatial Domain Access Constraint Use Constraint Time period of contents Status Contact Information Native Data Set Environment Repeated in other sections Contact organisation Contact person Contact position Contact address, telephone, facsimile, electronic , hours of service Person who is knowledgeable about the Data software and version, operating system and version, platform e.g. ArcCatalogue 9.2, Windows 2003

20 Identification Data Quality Spatial Data Organization Spatial Reference Entity and Attributes Distribution Metadata Reference

21 Data Quality General assessment of the quality of the Data Attribute Accuracy Report Logical Consistency Completeness Positional Accuracy Lineage Deductive estimates - Any estimates, even a guess based on experience is permitted. "Good", "poor" should be explained in a quantitative manner if possible. Test based on independent samples Test based on polygon overlay Assessments as to how true the attribute values may be. May refer to field checks, cross-checks with other documents, statistical analysis of values, and parallel independent measures. It does NOT refer to the positional accuracy of the feature. e.g. attribute accuracy checked by comparing hardcopy records randomly selected.

22 Data Quality Attribute Accuracy Report Logical Consistency Completeness Positional Accuracy Lineage check for bad values and ill conditions, e.g. only closed polygons are present in the Data. Is there anything that the Data does not included? e.g. landslides in natural terrain are based on aerial photos interpretation and no data is available between 1990 and 1992.

23 Data Quality Attribute Accuracy Report Logical Consistency Completeness Positional Accuracy Lineage Mandatory Source_Information: Source_Citation: Citation_Information: Originator: Planning Department, the Government of Hong Kong Special Administrative A description of the source material from Region which the data were derived. Publication_Date: 2001 Title: Street Block Geospatial_Data_Presentation_Form: plan The methods of derivation, including all Publication_Information: transformations Publication_Place: involved Hong in Kong producing Special the final digital files. Administrative Region Publisher: Planning Department Source_Scale_Denominator: 5000 to The Type_of_Source_Media: description shall include Hard copy the plandates of the Source_Time_Period_of_Content: source material and the dates of ancillary Time_Period_Information: information used for update. The Single_Date/Time: date assigned Calendar_Date: to a source unknownshall reflect the date that Time_of_Day: the information unknown corresponds to the Source_Currentness_Reference: ground; however, if this 2001 date in not Source_Citation_Abbreviation: None known, then a date of publication may be used, if declared as such. Source_Contribution: Spatial and attribute information Process_Step: Process_Description: By digitizing Process_Date: 2001

24 Identification Data Quality Spatial Data Organization Spatial Reference Entity and Attributes Distribution Metadata Reference

25 Spatial Data Organisation Direct spatial reference method Point and vector object Raster Object For Data with many objects, use ArcCatalog to get the count Mechanism to represent spatial information The objects used in represent space in the Data e.g. Point, Vector and Raster [Only one attribute in the domain is allowed] SDTS_Terms_Description: SDTS_Point_and_Vector_Object_Type: G-polygon Point_and_Vector_Object_Count: 4815 The types and numbers of vector or nongridded point spatial objects in the Data. The domain is defined in "Spatial Data Transfer Standard (SDTS)" document e.g. point, entity point, area point, label point; line segment, arc, link (topological linkage, network), chain; Interior area, G-polygon

26 Spatial Data Organisation Direct spatial reference method Point and vector object Raster Object Raster_Object_Type: Pixel Row_Count: 512 Column_Count: 512 Vertical_Count: 1

27 Identification Data Quality Spatial Data Organization Spatial Reference Entity and Attributes Distribution Metadata Reference

28 Spatial Reference Geographic Planar Local Description of the reference frame for horizontal position. It can be one of the following: Geographic, map projected, grid coordinates system, local planar Geographic Latitude and longitude resolution Geographic unit: degree, minutes, second Planar description should be used. Data owner to define the local coordinate system and define how this relates to the other projection

29 Spatial Reference Geographic Planar Local Planar: Map_Projection: Map_Projection_Name: Transverse Mercator Transverse_Mercator: Scale_Factor_at_Central_Meridian: 1.0 Longitude_of_Central_Meridian: Latitude_of_Projection_Origin: False_Easting: False_Northing: Planar_Coordinate_Information: Planar_Coordinate_Encoding_Method: distance and bearing Distance_and_Bearing_Representation: Distance_Resolution: 1 Bearing_Resolution: Bearing_Units: Degrees and decimal minutes Bearing_Reference_Direction: North Bearing_Reference_Meridian: Magnetic Planar_Distance_Units: Millimeters

30 Spatial Reference Geographic Planar Local Planar: Grid_Coordinate_System: Grid_Coordinate_System_Name: Other Grid System Other_Grid_System's_Definition: Hong Kong 1980 Grid Planar_Coordinate_Information: Planar_Coordinate_Encoding_Method: coordinate pair Coordinate_Representation: Abscissa_Resolution: 1 Ordinate_Resolution: 1 Planar_Distance_Units: Millimeters Geodetic_Model: Horizontal_Datum_Name: Hong Kong 1980 Geodetic Datum Ellipsoid_Name: International Hayford (1910) Semi-major_Axis: Denominator_of_Flattening_Ratio: 297.0

31 Identification Data Quality Spatial Data Organization Spatial Reference Entity and Attributes Distribution Metadata Reference

32 Entity and Attribute Information Entity Entity Type Label Entity Type Definition Entity Type Source Attribute Attribute Label Attribute Definition Attribute Source Attribute Domain Values Detail description - provide meaning of entity, attribute, and attribute value information associated with the spatial information Gives overview description if database documented in another form such as a data dictionary or data specification manual. PU - Planning Units PPU ID SPU ID TPU ID SBVC SHAPE Integer Integer Integer Integer Geometry

33 Entity and Attribute Detailed Description Entity_and_Attribute_Information: Detailed_Description: Entity_Type: Entity_Type_Label: PU Entity_Type_Definition: Planning Units Entity_Type_Definition_Source: Arc/Info Attribute: Attribute_Label: PPU_ID Attribute_Definition: Primary Planning Unit Attribute_Definition_Source: PlanD Attribute_Domain_Values: Range_Domain: Range_Domain_Value: Integer Range_Domain_Value_Definition: None Range_Domain_Value_Definition_Source: None

34 Entity and Attribute Attribute: Detailed Description Attribute_Label: SHAPE Attribute_Definition: Feature Geometry Attribute_Definition_Source: ESRI/Arcinfo Attribute_Domain_Values: Range_Domain: Range_Domain_Value: Integer Range_Domain_Value_Definition: None Range_Domain_Value_Definition_Source: None

35 Identification Data Quality Spatial Data Organization Spatial Reference Entity and Attributes Distribution Metadata Reference

36 Distribution Distributor Contact Information Resource Description Distribution Liability Standard Order Process Digital form Information about the distributor and how the data can be obtained by others Same structure as in Citation Section Leave blank,will be assigned later A statement describing the liability assumed by the distributor. e.g. deny liability if the data is incomplete, misused and incorrect; and limit the redistribution of the data by third party

37 Distribution Distributor Contact Information Resource Description Distribution Liability Standard Order Process Digital form Standard_Order_Process: Digital_Form: Digital_Transfer_Information: Format_Name: ESRI Shapefile Digital_Transfer_Option: Online_Option: Computer_Contact_Information: Network_Address: Network_Resource_Name: ginfo.cedd.hksarg Fees: Free of charge

38 Identification Data Quality Spatial Data Organization Spatial Reference Entity and Attributes Distribution Metadata Reference

39 Metadata Reference Metadata Date Metadata Contact Metadata Standard Metadata Security Metadata Extensions

40 Metadata Reference Metadata Date Metadata Contact Metadata_Reference_Information: Metadata Metadata_Date: Standard Metadata Security Metadata_Contact: [Refer to standard contact information] Metadata Extensions Metadata_Standard_Name: FGDC Content Standards for Digital Geospatial Metadata Metadata_Standard_Version: FGDC-STD Metadata_Time_Convention: local time Metadata_Security_Information: Metadata_Security_Classification_System: NA Metadata_Security_Classification: Unclassified Metadata_Security_Handling_Description: NA Metadata_Extensions: Online_Linkage: Profile_Name: ESRI Metadata Profile

41 Useful References Content Standards for Digital Geospatial Metadata Workbook Content Standard for Digital Geospatial Metadata Metadata Quick Guide Ten Most Common Metadata Errors Tkme

42 IMPORTANT Before confirming the metadata, some key aspects of the geospatial data shall be examined, corrected where necessary: spatial reference - is defined in HK 1980 grid and not in geographic consistency in spatial organisation - don't mix up points, polygons and annotations in one feature date and time format is "yyyymmdd" and local 24 hour time

43 Core Geospatial data

44 A1 - Images Core Geospatial Data Data Data Owner Data Agent Note Digital Ortho-rectified Images LIC/LandsD CGE/P 1963 B&W HKSAR 1: B&W Tsing Shan 1: B&W Tsing Shan 1: B&W HKSAR 1: /74 B&W HKSAR 1: B&W HKSAR 1: /94 Colour HKSAR 1: Colour Tsing Shan 1: Colour Tsing Shan 1: Colour HKSAR 1: Colour HKSAR 1: Infrared HKSAR 1: False HKSAR 1: Colour 2001 Colour HKSAR 1: Colour HKSAR 1: Colour HKSAR 1: 5 000

45 A1 - Images Core Geospatial Data Data Data Owner Data Agent Note Satellite Images CGE/P Selected satellite images in raw and geotiff format. 42 sets of images purchased between 1987 and 2002, including SPOT, LANDSat and IKONOS.

46 A2 - Digital Elevation Model Core Geospatial Data Data Data Owner Data Agent Note Digital Elevation Model (DEM) LandsD CGE/P 10 m grid size DEM supplied by the Lands Department 5 m grid size - based on 1:5000 LIC topographic maps, supplemented with ground elevation of boreholes and spot heights 2 m grid size - based on 1:1 000 LIC topographic maps, supplemented with government Catalogue of Slopes

47 A3 - Landslide Data Core Geospatial Data Data Data Owner Data Agent Note District Landslide Incident and Action Records GEO Landslide Data CGE/I, CGE/ME, CGE/MW CGE/LPM1 Data mainly contain information in the ECC-1 form related to reported landslides in the current year, recommended actions and classification of landslide incidents, etc. Data contain landslide information reported in the ECC-7 form and the Landslide Cards. Locations of the landslide were confirmed by LIC and District Engineers. Enhanced Natural Terrain Landslide Inventory (ENTLI) CGE/P Identification of ENTLI based on low-altitude (below ft) aerial photographs. The Data is an enhancement to the NTIL.

48 A3 - Landslide Data Core Geospatial Data Data Data Owner Data Agent Note Landslide Study Sites CGE/LPM1 Data contain the sites where landslide study has or had been carried out. Attribute table includes the feature no. Large Natural Landslide Data CGE/P Data in 1:5 000 scale contain the locations of large natural terrain landslides (with scar > 20 m wide). This was based on interpretation of 1963 low-level aerial photos (at ft) and review of NTLI. Natural Terrain Landslide Inventory (NTLI) CGE/P Digitized maps that show the source of natural terrain landslides and the travel path of the debris. The natural terrain landslides were identified by interpreting high altitude aerial photos (> ft).

49 A4 - Slope and Catchment Data Core Geospatial Data Data Data Owner Data Agent Note Catalogue of Slopes CGE/SS Data containing location of registered features in the Catalogue of Slopes. Attribute tables include basic information, such as slope geometry and its composition, consequence categories, etc. Slope Maintenance Responsibility LandsD CGE/SS Data contain the maintenance boundary of registered features and include the parties responsible for the maintenance. Natural Terrain Hazard Mitigation Measures Works (ND and NS Features) CGE/P Data contain the location of natural terrain hazard mitigation measures constructed in the territories.

50 A5 - Maps Core Geospatial Data Data Data Owner Data Agent Note Boundary of Scheduled Areas and Designated Areas Country Parks & Reserved Areas Demographic Data District Council Boundary CGE/I CGE/ME CGE/MW AFCD & Planning Department Census & Statistic Department Home Affairs Department CGE/P CGE/P CGE/SS Data defining the boundary of the Scheduled and Designated Areas in Hong Kong Boundary of country parks, reserved areas and site of special scientific interest. Data to be obtained from the data owners. Plans and Data for the 2001 Census is available. Data containing the geographic boundaries of the 18 district councils.

51 A5 - Maps Core Geospatial Data Data Data Owner Data Agent Note GEO District Boundary Lamp Post Marks Land Utilization Map Natural Terrain Hazard Studies CGE/I CGE/ME CGE/MW HyD PlanD CGE/P CGESS CGE/P Boundary of GEO District Divisions currently available in SIS. Data containing the reference number of lamp post in the territories. Plans that are provided by the Planning Department once every two years. Data containing sites where natural terrain hazard study have been carried out and mitigation measures proposed.

52 A5 - Maps Core Geospatial Data Data Data Owner Data Agent Note Outline Zoning Plans Utilities Tunnel Alignments (UNDER REVIEW) Planning Department Utility Companies, CLP and HKE CGE/P CGE/I CGE/ME CGE/MW Digitized outline zoning plans provided by Planning Department once every year. Data containing the alignment of tunnels and caverns constructed by private corporations and companies. Terrain Classification Maps (TCM) CGE/P 1:20000 TCM maps prepared under the GASP in 80s and contain three attributes, such as slope gradient, terrain component and erosion type. Topographic Maps Land CGE/SS Topographic survey maps in 1:1 000, Information 1:5 000, 1:10 000, 1: and Centre/LandsD 1:

53 A6 - Geotechnical Data Core Geospatial Data Data Data Owner Data Agent Note Aerial Photograph Flight Path Geochemistry Data Disused Tunnels LandsD CGE/P CGE/P CGE/P Data containing flight path and positions of aerial photographs taken. Coverage of the photos is computed. Data containing information on rock and stream sediment chemistry; and the surface chemistry data as reported in the Geochemical Atlas of Hong Kong. Data containing the location and alignment of disused tunnels in the territories.

54 A6 - Geotechnical Data Core Geospatial Data Data Data Owner Data Agent Note Geological Maps CGE/P 1: simplified geological map. 1: maps covering digitized data on solid geology, superficial deposits and major structural elements. Magnetic Survey Seismic Survey Map CGE/P CGE/P 1:5000 map for selected areas. Map showing the magnetic anomalies measured in the marine surveys. Data containing the track plots of marine seismic surveys for mapping offshore geology.

55 A6 - Geotechnical Data Core Geospatial Data Data Data Owner Data Agent Note Rock Sample Location Rainfall data (Historical and Realtime) CGE/P CGE/S&T Data containing location of rock samples collected by the Hong Kong Geological Survey. Data containing the coordinates of raingauges installed in the territories. Textual data include rainfall collected at every 5-minute interval year round.

56 A7 - Management of Projects Core Geospatial Data Data Data Owner Data Agent Note Checked Slope Status Slope Checking Status in Feature Status Review Private Development Sites CGE/I, CGE/ME, CGE/MW CGE/I, CGE/ME, CGE/MW CGE/I, CGE/ME, CGE/MW Data containing locations of features that have been checked by GEO after , including upgraded, newly formed, studied or removed features. Data include locations of active and completed private development processed by GEO. Textual data includes personnel approved fro qualified site supervision.

57 A7 - Management of Projects Core Geospatial Data Data Data Owner Data Agent Note Public Development Sites CGE/I, CGE/ME, CGE/MW Data include locations of active, completed and tentatively proposed public developments processed by GEO. DHO & Advisory Letters Action Sites CGE/I, CGE/ME, CGE/MW Data contain sites or slopes where DHO or Advisory Letter were issued. Inspected Dwellings CGE/ME Data containing locations of squatter dwellings that have been inspected by GEO. HD Squatter Control Areas CGE/MW Data containing boundary of the HD Squatter Control Units.

58 A7 - Management of Projects Core Geospatial Data Data Data Owner Data Agent Note NDC Study Boundary Inspected Dwellings Enhanced Slope Maintenance Records District Work Management CGE/MW CGE/ME Slope Maintenance Agents CGE/I CGE/ME CGE/MW CGE/I Data containing areas where nondevelopment clearance study has been carried out. Data containing locations of squatter dwellings that have been inspected by GEO. Data containing textual attributes that record the enhanced slope maintenance actions undertaken in man-made slopes.

59 A7 - Management of Projects Core Geospatial Data Data Data Owner Data Agent Note NDC Dwellings Land Matter Sites NT Exempted Houses CGE/MW CGE/I CGE/MW CGE/ME CGE/MW CGE/ME Data containing location of squatter dwellings where NDC recommendation has been made. Textual data contains detailed information related to the NDC. Data containing the locations of sites that are processed in the District Land Conference and been given a land allocation. Data containing the locations of NT exempted houses. GCC Sites CGE/I CGE/MW CGE/ME Data containing the locations of sites where GCC decision has been sought. Attribute table only contains relevant GCC paper No.

60 A7 - Management of Projects Core Geospatial Data Data Data Owner Data Agent Note Ground Anchor Sites Quality Supervision CGE/I CGE/I CGE/ME CGE/MW Data containing the locations of sites where permanent anchors are installed. Active Construction Sites CGE/I CGE/ME CGE/MW Planning Comment Records CGE/P Data containing the sites where planning comments have been provided to clients and other parties.

61 Core Geospatial Data What you need to do next? Examine the geospatial to check if it is up to the quality that you will specify in the metadata Compile the metadata Report progress quarterly: 1 April, 2 July, 1 September, 1 December 2008 Submit the metadata to SGE/S2 after signed off by data owner Target completion for compiling all the metadata : 31 December 2008

62 Core Geospatial Data What I will do? Create a data warehouse as central repository of core geospatial data Compile a web-based, searchable, data catalogue based on the metadata

63 Tools for Metadata Preparation

64 Tools for Metadata Preparation GIS desktop application : ArcCatalog 9.2 availability depends on license GIS desktop application : GEOMedia Catalog availability depends on license Standalone application : TKME freeware, though not very user friendly available in USGS website

65 Tools for Metadata Preparation ArcCatalog 9.2

66 Tools for Metadata Preparation TKME

67 Questions?