Corporate Spatial Data Infrastructure Standards

Transcription

1 Corporate Spatial Data Infrastructure Standards Version 2.4 Prepared by: Geomatics Yukon Department of Environment Department of Energy, Mines, and Resources Pacific GeoTech Systems North Inc. November 10, 2011

2 Table of Contents 1 REVISION HISTORY INTRODUCTION BACKGROUND OVERVIEW OF THE CORPORATE SPATIAL DATA INFRASTRUCTURE (CSDI) PROJECT PURPOSE OF THE CORPORATE SPATIAL DATA INFRASTRUCTURE IS SCOPE DEFINITIONS CSDI GOVERNANCE, PRINCIPLES, AND TECHNICAL ROLES CSDI Principles Data quality Data integration Access to data Data management TECHNICAL ROLES ICT Database Administrators (DBAs) ICT Network Administrators SDE Administrator RESPONSIBILITIES Use, Distribution and Support Responsibilities ORACLE/SDE DATABASE STANDARDS ORACLE STANDARDS SDE CONVENTIONS Schema Naming Constraints Feature Class Naming Role naming Standard Roles Oracle Roles for Specific Access User naming SPATIAL DATA STANDARDS STORAGE STANDARDS METADATA Metadata Standards WAREHOUSE STANDARDS AND PROCEDURES PROCEDURES FOR WAREHOUSE DELIVERY Warehouse Environment Warehouse Delivery Responsibilities Warehouse Delivery Naming Conventions Delivery Directory Structure Versions Directory Structure Detailed List of Delivery Tasks Warehouse Dataset Delivery Requirements Creation of application schema DATA PROMOTION STANDARDS Minimum Data Standards for Data Loading Criteria for inclusion Steps to prepare for data loading... Error! Bookmark not defined. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov10_2011.doc i

3 Data Model Standards DATA DEPICTION STANDARDS General Rules Symbology File Naming Standard (layer file) Layer Structure and Ordering Layer Title Naming Standard OPERATIONAL DATABASE STANDARDS OVERVIEW CURRENT OPERATIONAL ENVIRONMENTS GOVERNANCE OF THE NEW OPERATIONAL ENVIRONMENT SYNCHRONIZATION BETWEEN OPERATIONAL DATABASES AND THE WAREHOUSES Using a bridge (acquirer) Check-in/Check-out process SECURITY AND USER ACCESS ArcSDE Multi user Geo-database and Oracle Roles METADATA SPATIAL DATA STANDARDS APPENDIX A ORGANIZATIONAL CODES APPENDIX B: CORPORATE SPATIAL WAREHOUSE (CSW) LAYERS APPENDIX C: ACQUIRERS G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc ii

4 1 REVISION HISTORY This section of the document records various releases of this document. Initial releases will start with 0.1. The first approved published release will be 1.0. Release Date Author Details/Description 0.1 November PGTS North Inc. Draft 28, December 04, December 06, December 12, January 6 th January 17 th January 23 rd August 3, September 30, March 19, June 26, 2008 PGTS North Inc., Lauren Crooks, Stuart Alexander, Manon Desforges and Gerry Perrier Lauren Crooks, Stuart Alexander, Manon Desforges, Gerry Perrier and Eugen Toaxen Lauren Crooks, Stuart Alexander, Manon Desforges, Gerry Perrier, Diedre Davidson and Eugen Toaxen Olga Kopriva, Eugen Toaxen Lauren Crooks, Stuart Alexander, Manon Desforges, Diedre Davidson, Eugen Toaxen and Olga Kopriva Olga Kopriva Lauren Crooks Lauren Crooks Lauren Crooks Lauren Crooks Revised based on Lauren Crooks, Stuart.Alexander, Manon.Desforges and Gerry Perrier comments Added roles and responsibilities, new schema naming from Lauren, other edits Revised document Incorporate changes from version 0.2,3,4. Revised Operational, added delivery structure Data Standard review workshop. Changed recommendations to decisions. Incorporating changes from January 17 th workshop, creating appendix X Parking Lot attachment for decisions and samples appendix C Incorporating changes from implementation of Phase 1. Updating and ensuring consistency of terminology. Correcting grammatical and spelling mistakes. Updating for CSW2 (Geoconnections project) Incorporating standards developed during CSW2 needs approval of DAC before official Updating schemas and sde feature classes, feature class naming, appendix A 2.2 Dec 4, 2008 Lauren Crooks Updating schemas and sde feature classes, feature class naming, add TBL as a feature_geometry_type 2.3 October 25, November 9, 2011 Lauren Crooks Lauren Crooks Updating Appendices A and B Updating Appendices B and C G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov10_2011.doc 1

5 2 INTRODUCTION 2.1 Background 2.2 Overview of the Corporate Spatial Data Infrastructure (CSDI) project The objective of this document is to identify the minimum standards necessary for Yukon government to fully implement the Corporate Spatial Data Infrastructure (CSDI). This report sets the direction that will guide the Information & Communications Technology (ICT) Department and other related Yukon Departments through the implementation of the Corporate Spatial Warehouse (CSW) and Operational databases. There are several objectives of the CSDI implementation. When all the following objectives are met, the CSDI implementation will be considered complete: 1. A corporate spatial warehouse is created as a read-only ESRI Spatial Database Engine (SDE)/Oracle database; 2. Operational database environment(s) are created and configured that support day to day editing, maintaining and updating of each business area; 3. Relevant datasets are migrated and managed in an SDE Operational database within the CSDI infrastructure; 4. Clear standards have been defined for SDE layer naming, symbology, metadata requirements and access security; 5. Relevant datasets are migrated from existing formats and department environments to the warehouse; 6. Enterprise applications are in place to facilitate data access, management, and editing where necessary. Department-specific applications are migrated or their relevant functionality integrated into other business applications; and 7. Data is accessible to users via Web Map Viewer and/or GIS desktop tools. 2.3 Purpose of the Corporate Spatial Data Infrastructure is To maintain data integrity by limiting insert, delete, and update privileges; To protect sensitive or confidential information; To enforce copyrights and data-use agreements; To ensure that standards are followed; To guide the growth of the warehouse; To ensure that the warehouse contains appropriate data; and To serve clients in support of day to day operations; G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 2

6 2.4 Scope The scope of the Corporate Spatial Data Infrastructure is reflected in the range of services, content and data formats that are offered: Services o o o o o Metadata Distribution Access Population Support Types of Content o o o o Cadastre ( for example, tenures, ownership, boundaries) Resource Information (vegetation, fisheries, wildlife, ecology) Territorial Atlas (rivers, roads, buildings, topography, survey) Planning and Analysis Information (any planning units) Data format o o o o Tabular (attribute data) Vector (spatial data) Raster (images) Documents (associated with other CSDI content) The scope of the CSDI project is to: Establish a corporate repository for integrated land, resource and geographic data that supports a variety of business requirements for the natural resource sector, other government agencies, industry and the public; Provide unified access and discovery services to qualified clients with the ability to view and locate data from CSW; Provide clients with key information that supports their day to day operational needs such as data views, data downloads, data distribution and querying capability; Provide clients with data administration and population services; Encourage custodians to populate the warehouse with well structured, integrated and analyses-ready datasets; Establish the corporate spatial warehouse by consolidating and rationalizing existing operational datasets; and G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 3

7 Improve the relevance and value of existing spatial data by replacing data silos with data in the corporate spatial warehouse. 2.5 Definitions ArcSDE Layer An Oracle object/table that has been spatially referenced. An ArcSDE layer references an ESRI Feature Class and defines characteristics about how it is represented when displayed or printed. AXL File (pronounced 'axle file'). A file in XML format used by ArcIMS that stores the path to a source dataset and other layer properties, including symbology. Similar to a 'Layer File'. Business View A non-physical grouping of data for a specific business need. See also 'View'. Coverage A proprietary data storage format for storing geographic features using ArcInfo software. A coverage stores a set of thematically associated data considered to be a unit. In a coverage, features are stored as both primary features (points, arcs, polygons) and secondary features (tics, links, annotation). Feature attributes are described and stored independently in feature attribute tables. Coverage s usually represent a single layer, such as soils, streams, roads, or land use. Data Attribute A particular characteristic of a data entity for which information is to be collected. For example, weight and gender could be attributes of a Person' entity. Data Collection An aggregation of datasets or similar data. Data Set A managed grouping of similar business data with a single data custodian. The data that results from and is maintained by a distinct set of processes within a business area. Examples include Base data, Geology data, Forestry data, and Mineral data. Data Warehouse A collection of software and data organized to collect, cleanse, transform and store data from a variety of sources, and analyze and present information to support decisionmaking, tactical and strategic business processes. Feature An abstraction of a real world phenomenon; it is a geographic/spatial feature if it is associated with a location relative to the Earth. A feature is a single instance of a feature class. Feature Class A collection of geographic features with the same geometry type (such as point, line, or polygon), the same attributes, and the same spatial reference. Feature classes can be defined in Geo-database, shape files, coverages, or other data formats. Feature classes allow homogeneous features to be grouped into a single unit for data storage purposes. For example, highways, primary roads, and secondary roads can be grouped into a line feature class named "roads." Group Layer G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 4

8 A group of layers that appear and act as a single layer. Group layers make it easier to organize a map, assign advanced drawing order options, and share layers for use in other maps. Layer The visual representation of a geographic dataset in any digital map environment. Conceptually, a layer is a slice or stratum of the geographic reality in a particular area, and is more or less equivalent to a legend item on a paper map. The physical grouping of a geographic data set somewhat equivalent to a grouping of (a) feature class. Also known as 'Map Layer'. See also 'ArcSDE Layer'. Materialized View A table which is automatically updated based on a pre-defined view definition and update interval. Materialized views have two usages: for replication and for performance. See also 'View'. Presentation View A grouping of data with display properties, usually based on a specific business need. See also 'View'. Spatial View Spatially enabled view of feature classes that may or may not be combined with attribute data. Theme 1) A business grouping of data with one or more custodians. 2.) A complete set of layers with unique symbology bundled together to meet a specific business requirement in a mapping application. 3.) In ArcView 3.x, a set of related geographic features such as streets, parcels, or rivers, along with their attributes. All features in a theme share the same coordinate system, are located within a common geographic extent, and have the same attributes. View A means of accessing a subset or superset of data. See also 'Business View', 'Database View', 'Materialized View', 'Presentation View', 'Spatial View'. 3 CSDI Governance, Principles, and Technical Roles CSDI governance is described in a separate document, entitled Corporate Spatial Data Infrastructure Governance. Governance describes decision making authority. Technical roles and responsibilities for implementing and maintaining the CSDI are described below in Section CSDI Principles The following are the principles under which the CSDI will operate Data quality The corporate spatial warehouse should contain data of (i) the highest quality available based on business needs; and (ii) of known quality that is documented in metadata. There may be cases where data availability is more important than data quality; in those cases, the warehouse should contain such data. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 5

9 3.1.2 Data integration Full spatial integration of all data is a long-term goal. In the meantime, proper registration/alignment with Data Advisory Committee approved bases, used in the warehouse is the next best thing Access to data Data Custodians (defined in Corporate Spatial Data Infrastructure Governance) have ultimate authority on access. Access privileges must conform to Access to Information and Protection of Privacy (ATIPP) legislation, copyright, sensitivity issues, etc Data management All data editing occurs in operational systems outside of the warehouse; Data redundancy should be avoided; and Data subsets are best created with Views 3.3 Technical Roles ICT Database Administrators (DBAs) Application Services staff at ICT are responsible for: Assisting Geomatics Yukon with maintaining a stable CSDI Environment; Developing, maintaining and monitoring CSDI Oracle database services; Managing Oracle accounts and user groups; and Backing up the Oracle databases ICT Network Administrators Assisting Geomatics Yukon with maintaining a stable CSDI Environment; and Building and maintaining the CSDI infrastructure server hardware and network SDE Administrator The SDE Data Administrator is responsible for: Maintaining a stable CSW SDE Environment with support from other Geomatics and ICT staff; Providing operational SDE support; Developing, maintaining and monitoring spatial data services (i.e. ArcSDE services,); and Making sure population processes (acquirers) from the Operational repositories to the Warehouse are reliable and secure. 3.5 Responsibilities Use, Distribution and Support Responsibilities 1. Custodian must establish consistent rules for distribution of data to client groups. 2. Security profile is mandatory. 3. Information on appropriate use must be published in metadata repository. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 6

10 4. Support to users questions must be available via help desk 4 Oracle/SDE Database Standards 4.1 Oracle Standards The Department of Environment Oracle standard is adopted for the CSDI: YEIS_Standards_v2.doc. 4.2 SDE conventions Schema Naming Schema can include data from more than one custodian and include multiple scales. Schema should group data from similar subject areas. This is the dominant factor. The Oracle schema will be used as a subject area for classifying or grouping the data as determined by the Data Advisory Committee. Appendix B shows the Data Classification Hierarchy. The first level is a broad subject area grouping and will not be reflected in ArcSDE (however it will be used in Metadata, Layer files, and presentation services). The second level is the subject area and will be used as the actual schema name. The third level is the SDE feature class name (data layer) Constraints 1. Oracle physical table generation restricts names to 30 characters; therefore, SDE feature class names may be up to 30 characters long. The schema name is not included in the 30 characters, just the prefix, descriptive name, and suffix. To allow for the creation of spatial views with the same name as the base table, feature class names should be restricted to 26 characters except for those feature classes that will never have spatial views (e.g. rasters). 2. Feature class names must start with a letter, and may contain only letters, digits, and '_' (underbar). 3. Names cannot contain blanks; therefore, words within the name will be separated by '_' (underbar). 4. Oracle table names are case-insensitive. Although names can be entered and queried in upper or lower case, they will be stored and compared in UPPER CASE only. 5. SDE includes a layer description field, which is limited to 63 characters. This is a free text field, which may contain blanks and other punctuation Feature Class Naming Org Code>_<Intuitive Descriptive Name >_ <Version>_<Feature Geometry>_<Compilation Scale> G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 7

11 Org Code: section. OPTIONAL Refer to Appendix A for codes. Intuitive/Descriptive Name section. MANDATORY. This section must contain a descriptive name, with words separated by '_'. The intention is that the name should be as intuitive of the information the layer contains as is possible, e.g. GAME_MGMT_SUBZONE. The name should be singular. Feature Classes that are identical at different scales, must have identical names except for the compilation scale section. Base table names and spatial views should be identical. For feature classes that have imagery or raster feature geometry type, the resolution of the data is contained within this section after the descriptive name. For imagery data, either satellite or ortho is also contained within this section after the resolution. Example: BLUE_MARBLE_500M_SATELLITE_IMG. This section of the layer name could also contain a code for the geographic extent of the layer data if it is needed to distinguish the dataset. This is a mandatory field. Version section. OPTIONAL. Versions are represented by dates. If version is used, the Year is mandatory. Format is YYYYMMDD. The capture date is optional and should be only used if required for name uniqueness. Month naming convention JA FE MR AP MY JU JL AU SP OC NO DE Feature geometry type section. MANDATORY (5 CHAR) POLY PT LN LLN ANNO IMG G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 8

12 RSTR COGO TBL The exception is for POLY: where AREA is part of the descriptive name, it is optional. E.g. PARK_PROTECTED_AREA. Satellite or orthoimagery has the feature geometry type IMG. Other raster data like DEMs, shaded relief, classed data has the feature geometry type RSTR. LLN refers to leader line (associated with annotation) TBL refers to a table with no geometry (as in NRN) Compilation Scale section. MANDATORY when the data have been compiled against a particular scale of base data. An exception, where this is NOT mandatory, is for raster (RSTR) and image (IMG) feature classes. Rather data resolution is contained within the intuitive/descriptive name. 10K 25K 50K 250K 1M 2M Role naming Standard Roles A standardized security authorization service should be developed as part of the CSW project. It is recommended to create a hierarchical role based security infrastructure. Each layer in CSW is one of the three roles: (IMS_SELECT); Government (CSW_YNET, schemaname_base); and Special Access. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 9

13 Government Special Access Special Access Special Access Special Access Special Access Special Access Special Access roles can be used to control the access to subsets of layers (e.g. by department or business area, data owner or custodian, data set version, etc.) to end-users or groups of end-users. The sensitive layers need to be identified and flagged in the Discovery and Metadata review process. Database accounts will fall into the following categories: 1. Data administration; 2. Web applications; Each Web application will require two unique proxy accounts (application Oracle users) (i) A development proxy account for use by the developer. This account will be locked and password reset when the new or updated application is promoted to production; and (ii) a production proxy account to be utilized by the application when it is promoted to production; haven t implemented this should we? 3. Custom client server application (requiring individual database accounts); and 4. Direct-connect analytical user (government staff or government sponsored) using tools like ArcGIS and Oracle SQL Oracle Roles for Specific Access In general, when a role is defined in the corporate warehouse, these rules must be followed:» When possible, must be defined hierarchically. This means that any role must be based in any other role with lower hierarchy, unless the role is intended to be used with the most basic grants and there is no role with a lower hierarchy.» Must be related to a data custodian/steward or data classification schema name unless the role is intended to be used for database administrative purposes;» Must not include any Oracle - specific roles, such as RESOURCE or CONNECT For role naming, the following standards are suggested: G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 10

14 Use only the characters A-Z and underscores (_). Use a maximum of 30 characters, including underscores. Use the following naming convention: CSW_<Data_Custodian_short_name>_<Security_level> Where: Data Custodian/Classification short name: Up to 17 characters. It defines the short name of the data classification schema name as it is defined in the database or intended Data Custodian/Steward name. In the cases in which a specific role to access the information belonging to different data classification schemas is required, the short name must describe as much as the purpose of the role. However, the creation of specific roles must be avoided, concentrating the efforts in a better distribution of the layers in different data classification categories. Examples: - If the data classification schema name is CSW_ADMIN_BOUNDARIES, its short name could be CSW_ADMN_BOUND_ Security level: ADMIN MGR USER BASE xxx This portion of the role name is intended to describe the purpose of the role. Below is a list of descriptive words that can be used to describe specific aspects of a role. Highest role assigned to an entire business dataset corresponding to a specific data custodian/steward. There should be only one ADMIN role for each data classification or business area. If there is a need for two different roles that have similar but different administration needs, the MGR name should be used instead. Highest role for a portion of an application or business area. MGR roles are junior in privileges to the ADMIN role Least privileged role used by a functional area. May require the base role. Due to the multiple uses of the term user, adding additional description to the role name is mandatory (e.g. CSW_MVB_DRIV_EXECUTIVE_USER) Least privileged role for an application or functional area; replaces roles such as READ_ONLY, SELECT and IMF used in operational databases. A base role must never have child roles. Lower levels of security (business area or individual business specific functions) should be described using a descriptive name (e.g. CSW_MVB_VEH_ACCOUNTS_CLERK). The purpose of the role type is to make the administration tasks easier. When a database has many roles, it is difficult to determine which role is associated with a particular purpose. With the role type and the description associated in addition to the data G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 11

15 custodian, the number of roles that need to be investigated is reduced and will be easy to find User naming Each user will connect to SDE with a unique username. The required roles will be granted to that user. The username will be identical to the user s YNET account name (eg jdbrown). 5 Spatial Data Standards 5.1 Storage Standards Projection Territorial datasets will all be in the Standard Yukon Albers projection. The coordinate units should be meters. Standard Yukon Albers Projection Projection Albers Equal Area Conic Datum NAD83 Spheroid GRS80 Central Meridian " W ( in decimal degrees) Projection Origin " N (59.0 in decimal degrees) 1st Standard Parallel " N ( in decimal degrees) 2nd Standard Parallel " N (68.0 in decimal degrees) False Easting 500,000 m False Northing 500,000 m Each dataset is assigned the same ArcSDE projection parameters. Datum transformation between NAD27 and NAD83 MUST use the Canadian National Transformation matrix. Accuracy and Precision/Resolution Feature classes loaded prior to ArcGIS 9.2 Unless there is a clear need for sub-meter resolution or where the area of polygon exceeds the maximum storage size, datasets within the CSDI should be stored in single precision coordinates (i.e. the precision in SDE should be set to 1000). This is sufficient for data used by the Yukon Government, as Albers (or even UTM) projection coordinates can be stored in single precision with meter accuracy. Feature classes loaded at ArcGIS 9.2 or later All data sets will be loaded with double-precision. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 12

16 Data Registration Where possible, data should be registered to existing topographic or planimetric data. The largest scale data with full coverage of the Yukon is the 1:50,000 NTDB, which, has been planimetrically corrected by NRCAN Center for Topographic Information, Sherbrooke (CTIS) using more accurate ground control points (correction matrices) to enhance the geometric accuracy of the base data (CTI, 2005). For smaller scale mapping, the NTDB 1:250,000 Scale Base Mapping has been adopted as a registration base for many projects within the Yukon Government. 5.2 Metadata Metadata or "data about data" describe the content, quality, condition, and other characteristics of data. Every dataset must have metadata attached to it. Metadata is information about a dataset, such as who is responsible for it, when it was created, what its source was, its spatial extent, its accuracy, etc. The Yukon government follows the FGDC standard for capturing information. Further information on the FGDC standard can be found at the following URL: Metadata Standards Metadata management is an important part of data warehousing. Current, descriptive metadata adhering to content standards should be maintained for all shared data sets. The following table describes the minimum Corporate Metadata service elements that need to be populated before publishing to METADATA server. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.4_nov8_ doc 13

17 Corporate Metadata Service Elements Sections Identification Tabs Categories Types Schema Name Description Comment General Citation Title Follow the naming convention. This section is Mandatory used by other sections of the metadata standard. Originator This is the person or organization that created the Mandatory data set. If the name of editors or compilers are provided, the name must be followed by "(ed.)" or "(comp.)" respectively. ation Date The date that the dataset was published or otherwise made available for release. Mandatory Title The name by which the data set is known. Mandatory Time Period of Content Status Point of Contact This is the person and organization to contact for more information about the data set. This may be the same as Originator. Person The person, and the affiliation of the person, associated with the data set. Used in cases where the association of the person to the data set is more significant than the Association of the organization to the data set. Organization The organizational unit name of the custodial organization. Custodial responsibility lies with the Director or most senior official. Abstract A description of the data set. This field is very important since during internet searches all that is returned is the title and the abstract. Purpose A summary of the intentions with which the dataset was created. Mandatory Mandatory or Optional (should be mandatory? - to discuss at the workshop Optional Mandatory Mandatory Access Constraints Identifies the access constraints for the dataset. Mandatory Use Constraints Identifies the use constraints for the dataset. Mandatory Information about the date and time of an event. Mandatory Currents Reference Select from the dropdown list. Mandatory Default= publication date. An indication of the status of the dataset Progress The state of the data set. Mandatory Default = complete. You must use the dropdown list. Maintenance and Update Frequency A description of the frequency with which changes and additions are made to the resource after the initial resource is completed. Default = none planned. Bounding Coordinates These must be entered as decimal degrees. Mandatory G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 14

18 Keywords North Northern-most coordinate of the limit of coverage expressed in latitude. South Southern-most coordinate of the limit of coverage expressed in latitude. East Eastern-most coordinate of the limit of coverage expressed in longitude. West Western-most coordinate of the limit of coverage expressed in longitude. Mandatory Mandatory Mandatory Mandatory Metadata Reference Theme Keyword Thesaurus Place Keyword Thesaurus Stratum Keyword Thesaurus Temporal Keyword Thesaurus Common-use word or phrase used to describe the subject of the data set. Reference to a formally registered thesaurus or a similar authoritative source of place keywords. Reference to a formally registered thesaurus or a similar authoritative source of stratum keywords. Reference to a formally registered thesaurus or a similar authoritative source of temporal keywords. Mandatory Mandatory if applicable Mandatory if applicable Mandatory if applicable Keywords Words or phrases summarizing an aspect of the Mandatory or Optional data set. Tabs Categories Types Description Comment Metadata Date Follow date convention. Default = 2001/01/01 Mandatory Standard Name Metadata Standard Name FGDC Content Standard for Digital Geo-spatial Metadata Mandatory Standard Version unless you are importing from another source, Mandatory use the default = 2" General Contact This is the contact for the metadata information (not the spatial data, although this may be the same person). This field references the same list of contacts that is used for the identification section. Mandatory or Optional Person The Name of the person, select from the list on the left or add a person name if required. Contact The telephone number by which individuals can Telephone speak to the Address Mandatory Mandatory organization or individual. Mailing Address default = mailing address. You must use the Mandatory dropdown list. City The city of the address. Mandatory Province or State The state or province of the address. Mandatory Postal Code The ZIP or other postal code of the address. Mandatory G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 15

19 6 Warehouse Standards and Procedures 6.1 Procedures for Warehouse Delivery This section provides a set of standards and guidelines for the delivery of applications and `datasets into the Corporate Spatial Warehouse. The goal of documenting delivery standards is to facilitate a smooth and orderly delivery process. By following these standards and guidelines, better organization, communication and streamlined dataset loading will be achieved Warehouse Environment There are 3 warehouse instances each providing a separate function in the application delivery and migration process. Name CSW Delivery CSW Test CSW PROD Function Data compilation, initial data verification User acceptance testing Production server for internal and public access Warehouse Delivery Responsibilities The delivery of an application rarely requires the talents of a single resource, although an individual may perform many roles. The following is an explanation of the roles played by team members in the warehouse delivery process Application Administrator Responsibility - Business Area: Responsible for the ongoing operations of the application; Co-ordinates with the Business Analyst to schedule delivery of new versions or releases; Responsible for the User Acceptance Test (UAT) process; and Verifies that the application functionality adheres to government standards Delivery Personnel Responsibility - Vendor or Designated Business Area Staff: Vendor or business area staff members responsible for delivering applications to the government; Must designate one individual as the primary contact; and Responsible for running all scripts in delivery environment G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 16

20 ICT Database Administrator Responsibility - ICT: Provides administrative support for the government's Oracle based applications; Responsible for ensuring a stable application environment is available for the delivery of corporate applications; Normally performs all of the Database Administrator (DBA) tasks associated with a warehouse delivery; and Vendor/Business Area Project Manager Responsibility - Vendor/Business Area: Primary vendor/business area contact for the application; Responsible for the delivery process; and Responsible for ensuring that the application delivery adheres to Yukon government standards Geomatics Business Analyst Responsibility Geomatics Yukon: Coordinates all communication between the government staff and the vendor; Ensures that Yukon government standards and guidelines are followed for the application delivery; and Will co-ordinate all of the ICT resources required for the delivery process. Responsible for performing all aspects of the migrations to test and production (except for data conversion) Warehouse Delivery Naming Conventions Naming of files prior to 2010 Each delivery is considered a patch and is stored on ict-data on hunter:\gy\csdi\csw\csw_patches Naming conventions must be followed for all application deliveries; all files delivered must be in a compressed archive format using the naming convention shown below: <dataset name>.<version>.zip The <dataset name> is representative of the dataset or related application acronym. The <version> portion is the assigned version number for this delivery dataset zip First Release of "dataset". dataset zip Patch release for "dataset"v1.0.1 dataset zip New minor release of "dataset". G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 17

21 dataset zip Patch release for "dataset"v1.1.1 dataset zip Patch release for "dataset"v1.1.2 dataset zip New major release of "dataset". The versions are determined as <major release>.<minor release.<patch>. To constitute a major release the data must have "fundamentally changed". For a minor release, the database must have been altered significantly. For example, deleting or inserting a table, view, trigger etc. A patch would be delivered when there are only small updates Naming of files from 2010 Each delivery is considered a patch and is stored on ict-data on hunter:\gy\csdi\csw\csw_patches. A folder should be created for each schema and patches should be stored under this folder. The name will be the same as the schema name without the CSW prefix. Patch releases will follow the patch release naming convention above, but should be unique for each schema Delivery Directory Structure An application or dataset specific directory should be created in the delivery base directory for each new application. This can be coordinated through the ICT Delivery Specialist. In the home directory for each application, the following directories will need to be created by the ICT Delivery Specialist prior to the application delivery: Directory Name /delivery /test /prod /case Description Delivery instance Test Instance Prod Instance Location for all Data Model export G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 18

22 Instance Directory Structure For each instance of delivery, test and prod there will be a series of sub-directories created by the ICT Delivery Specialist prior to the initial application delivery. The structure of these sub-directories is as shown below: Sub-Directory Name /admin /scripts /docs /versions Description Scripts that will be used on a regular basis for admin purposes. Scripts that will be used once on delivery i.e. database objects, patch scripts, and SDE scripts All documentation regarding all datasets, including the initial readme file delivered. Version source directory Versions Directory Structure The versions directory is an inventory of all versions related to the datasets loads. For clarification, the <version> stated below is of the format ##.##.## where the first '##' represents a major release, the second '##' represents a minor release, and the third '##' represents a patch release. The files get copied into the corresponding <instance>/source directory as part of the delivery. Version # Sub-Directory Name Description <version> Version number that should be created when the delivered file is unzipped. I.e. Initial release of an application should be /admin Scripts that will be used on a regular basis for admin purposes. /scripts Scripts that will be used once on delivery i.e. database objects, patch scripts, and SDE scripts /docs All documentation regarding all datasets, including the initial readme file delivered. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 19

23 Detailed List of Delivery Tasks Task Schedule delivery with ICT/Geomatics Yukon; Verify the delivery scripts at the development (vendor's) site; Create the delivery kit following the delivery structure and compress with zip; Transfer the delivery kit to the ftp delivery server <dataset>/delivery/source/versions directory or directly to delivery location on ict-data on hunter; Extract the files being delivered from the delivery kit; Install the new version of the dataset/application following the instructions in the readme file; Revise the readme instructions if any errors or discrepancies are encountered; Verify the dataset and/or functionality of the application; Notify the Business Analyst and Delivery Specialist that the delivery has been completed Notify the Delivery Specialist if the delivery includes a CASE dump file. Migrate to test instance Usability Test Sign-off Migration to Test Migration to prod Final Sign-off Responsibility Project Manager Delivery Personnel Delivery Personnel Delivery Personnel Delivery Personnel Delivery Personnel Delivery Personnel Delivery Personnel Project Manager Project Manager Geomatics Business Analyst Application Administrator Application Administrator Geomatics Business Analyst Application Administrator Warehouse Dataset Delivery Requirements In the CSW the datasets reside in shared CSW_ schemas such as CSW_BASE or CSW_; there are some tasks associated with Warehouse dataset deliveries that must be completed. These include: The Vendor Delivery Personnel must contact the Geomatics Business Analyst to request table space names, role names and SDE configuration keywords to be used in the scripts if not known prior to the delivery; Output from the scripts must be spooled to a file; in case of failure this output will be sent back to the Vendor/Business Area Delivery Personnel for review; Data loading should be done via scripts containing insert statements. If this is not a valid method for data delivery the Vendor Delivery Personnel must contact the ICT Business Analyst to discuss other options; Scripts must be provided to assign the necessary grants to roles on the new objects; G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 20

24 Scripts must be provided that uninstall the new objects. The scripts must contain statements that perform uninstall at both the object and SDE metadata levels. These will be used when the objects have to be re-created; An exception to the standard delivery process is scripts are not required for the table space and data file creation or the schema creation; All data to be loaded into the CSW must be modeled and approved by the Data Administrator. After approval the data model must be loaded in the data model repository; Application CASE Dump files should be left in the.../versions/<version>/case directory and notification sent to the Delivery Specialist that the delivery includes an Oracle CASE dump file. The Delivery Specialist will then copy the dump file into the application's <app>/case directory and notify the Data Administrator, who will then load the dump file into the ICT CASE Repository; Any large scale data conversion or loading from other systems is the responsibility of the Vendor/Business Area Delivery Personnel on all the instances. It is also the Vendor's responsibility to ensure that new statistics are generated after the data load process; and Each delivery must include a readme.<version>.txt file that resides in the <version>/docs sub-directory. The readme file should include complete directions on how to install the version of the dataset/application being delivered. If the delivery is a patch release, it should also include what changes are actually being made to the database. Any changes to the database should be described down to the object level. For example listing any triggers, functions, procedures, and packages that are updated, or table changes and data conversions that are performed. The readme provided will include instructions surrounding any foreign keys that may need to be disabled if the object will be utilizing replication in the CSW Creation of application schema Some datasets are used in a context of an application and require application specific tables. Here is a list of requirements for handling application related objects: In CSW the application schema name must be named APP_<appname> where <appname> is the assigned application acronym. Relevant system privileges must also be granted to this schema. Scripts must be provided to Delivery Specialist to run prior to delivery; Creation of the application objects, public synonyms, etc.; Creation of application PROXY account and its associated privileges. This account must be created for any application that accesses the CSW, access is not permitted via the application owner. This account must be PROXY_<appname>. If the application has replication requirements these must be communicated to the Delivery Specialist prior to the delivery. Specific steps associated with replication must be completed as part of the delivery; Creation of user account in the source database for access from the CSW over a database link. The account name must be DBLINK_<appname> where <appname> is the assigned application acronym; Creation of materialized views; these objects must have a _MVW suffix so they can be easily identified; G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 21

25 Refresh type, frequency and schedule must be determined and approved the Delivery Specialist; Snapshot logs created in the source database; and Any materialized views involving complete refresh (i.e. no snapshot logs on the source database) must be approved by the Delivery Specialist. 6.2 Data Promotion Standards Data promotion to the corporate warehouse is a collaborative effort between the data custodian and the warehouse administrator. Data promoted to the CSW must comply with the Yukon government standards. As many types of information, spatial and attributes, hosted in the existing operational systems has value to a broad audience, it will be promoted to CSW. The CSW will formalize promotion process in order to achieve: Content Management; Acquirer and replication services; and CSW operational services. In support of the promotion standards, some physical requirements should be noted: CSW must have sufficient disc capacity; Server performance must be adequate to handle the traffic; and Access to CSW must be secured Minimum Data Standards for Data Loading Criteria for inclusion The data subject area has value to wide audience and/or has significance if it can be integrated with other land and natural resource data; Supports government resource management, planning or environmental monitoring activities; Has a designated custodian (Data Custodian Guidelines and Policies must exist in order to create the framework and to assure that data is managed based on both custodian business needs and user requirements); There is commitment to ensure continual refreshing of the data into the warehouse; Custodian can attest to fitness of data (e.g. model, quality assurance processes, metadata, updates, accuracy, etc.); Minimum geographic extent is captured; Is fully described in the Department s metadata repository (all data to be published must be described in the Metadata Repository to Department metadata standards. Metadata includes a description of fitness for use, the purpose for which the data is intended, and restrictions on use or modification. Data administrator has a support role in assisting the custodian in capturing metadata); Is well modeled, with its data model in the Department s Oracle design repository (data models may be developed based on models of operational systems or, on approval of the Data Administrator, as-built in an existing data set. Where transformations are required to present the data in an integrated fashion or to eliminate parts of the data set that may have redundant overlap with a better-sourced G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 22

26 data set, these transformations need to be addressed in the data model. Data administrator has a support role in the development of data models); Has appropriate security defined. Security definitions need to include both the description of security requirements; i.e., a description of limits or restrictions on access, and any authorization mechanisms, such as Data Use Agreements, and the implementation, i.e. what system user groups will have what permissions, and any authorization mechanisms. Security definitions should include any restrictions related to freedom of information and restrictions to distribution); and There is commitment to ensure continual refreshing of the data into the warehouse Data Model Standards Data modeling refers to the process of creating a description of the data classes (e.g. database tables) within a data set, their properties (columns/attributes) and the relationships between these data classes. A data model includes a blueprint describing the data classes and relationships between data classes for human consumption and a data dictionary for use by the CSW services. The warehouse requires two data models, one describing the structure of the data in the operational system and another describing the structure of the data in the warehouse. The operational data model is used to define the warehouse data model and the publication process, which is how the data is published to the warehouse. The warehouse data model is also used by the services to automatically interact with the data and can be used by the users of the data to understand its structure. A. Logical Naming conventions In order to create appropriate descriptions for the objects in a well defined data model, these rules must be taken in consideration: An object s description must be: Meaningful: Use of one to four real words. When the entity name is only one word, it must not be an Oracle reserved word or a word that is not self-explanatory. E.g. the words ACTION, RANGE or PERMIT; Self-documenting: By looking at the name, the reader should have a good idea what the name means without having to read the description); Derived from the business use or purpose but understandable to a non business user; Abbreviations or acronyms should be used only where no combination of names would produce an acceptable result. They are to be avoided unless universally understood; Entities and attributes do not contain _ (underscores); If the entity represents a code table it has the suffix CODE; and If the entity represents a cross-reference table it has the suffix XREF. G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 23

27 Relationship naming Relationship names must be meaningful; Both sides of a relationship must be named; and Phrases as 'has' 'is' 'related to', 'associated with' should be avoided in favor of more descriptive names. Descriptions for Entities The description of an entity must be clear, concise and understandable, following these rules: The first sentence of the definition should give a noun, or group of nouns, with modifying adjectives or phrases which summarize the meaning of the entity; In the cases in which an entity has an acronym in its definition, it must be fully qualified, e.g.. Universal Transverse Mercator (UTM); Examples of the information contained must be included. The first example should be one which is firmly in the collection, which will help the reader s understanding; Business intent of entity must be included. Where possible, the intent of the entity should be shown. This allows the reader to understand the rationale for the entity and fit their own understanding of the business into the one represented by this entity; and The descriptions for entities should not contain verbs that reference physical objects or mention of the physical database object. Descriptions for Attributes The First sentence summarizes the attribute description. It must contain a noun, or group of nouns, with phrases which clarifies the meaning of the attribute. e.g.: An EMPLOYEE ID is unique identifier for each employee, who works or has worked, for the company; Example should be included. The first example should be one which is firmly in the collection, which will help the reader s understanding. Other examples should reveal the bounds of the attribute; and Differentiation: Where there is confusion between two similar attributes in an entity, either because their names are similar or because their descriptions are similar, there should be an explicit text explaining the difference. Descriptions for Views The views need to be described just like entities and their attributes; and If the view in the current data model uses and object from another data model, that object and column must be indicated in the column description. B. Physical Naming Conventions The conventions to define spatial and database objects related to the spatial layers using Oracle Designer, such as columns, sequences, views, triggers, indexes and constraints shall follow these basic conventions: Any object name should be a maximum of thirty characters long; Database names are limited to 8 characters. However, the database links names can be defined up to 128 characters and can contain periods (.) and at (@) signs; G:\GY\CSDI\Documents\Standards\Standards document\csdi_standards_v2.3_oct25_2010.doc 24