Data Migration Plan for MIRPS Data and Reference Table Data

Transcription

1 Resource Ordering and Status System (ROSS) Program Professional Development Services Data Migration Plan for MIRPS Data and Reference Table Data August 16, 1999 Contract: GS-35F-4863G Delivery Order Number: GA51 Lockheed Martin Reference Number: GA510902

2 Table of Contents 1. INTRODUCTION MIRPS DATABASE MIGRATION PLAN ROSS DATA ITEMS REQUIRING VALUE INITIALIZATION DATA SOURCE INITIALIZATION METHOD(S) SPECIFIC SUPPORT SOFTWARE REQUIREMENTS DATA VALIDATION PLANNING Tasks Completed Initialization Method (2.3) Database Validation Execution of Migration DEVELOPMENT ENVIRONMENT DATABASE MIGRATION DEVELOPMENT DATABASE VALIDATION DEVELOPMENT DATABASE MIGRATION OPERATIONAL DATABASE VERIFICATION Appendices APPENDIX A. MAPPING OF MIRPS DATA TO ROSS... A-11 i

3 1. Introduction By the end of the Phase 3 task order, all reference data will have been loaded into the Vision JADE modeling tool as a part of the rules set instantiated on the Business Logic Server. This reference data consists of all the valid values constraints defined to date for ROSS. These value sets are available in a collection of tables supporting JADE. Due to this situation, no specific plan is necessary to load the reference data. For this reason the focus of this plan is the movement of MIRPS data to ROSS. The database migration plan for MIRPS has been developed by analyzing the SILVERRUN ERX Conceptual Data Model, the Data Dictionary for the Multi-Agency Incident Resource Processing System (MIRPS) Project, and the Analysis and Discussion Related to the Use of the Multi- Agency Incident Resource Processing System (MIRPS) on a Nationwide Basis. Appendix A depicts the mapping of various MIRPS data items to ROSS data items. The MIRPS data migration process will require multiple passes due to the differences in the MIRPS and ROSS schemas. Some MIRPS tables will translate on a one for one basis into ROSS. Others will require a more complex approach using specific dumps of MIRPS data acted on by SQL*Loader to fill the corresponding ROSS tables. 2. MIRPS Database Migration Plan The following procedures were followed in developing this plan: Determine which of the ROSS data items require value initialization. This will be affected by the data requirements defined in the test plans being developed during Phase 4, Application Build. Determine the source of the data, including any dependencies on organizations both internal and external. Determine the schedule for the data value initialization based upon the phased test plans and the phased software module delivery schedule. Also, determine the schedules for the receipt of any required external data. Determine the method of initialization, e.g., manual editing, running a special purpose program. Determine the requirements for any migration software necessary to support the process. This may include programs to transfer data from external format to the operational database, compare programs, special and general purpose editors. Determine how the data will be verified upon transfer to ROSS. Compile a database migration plan for the entire effort that defines schedules and responsibilities as well as resource requirements, including computer time, personnel and disk storage. Planning Results 2.1 ROSS Data Items Requiring Value Initialization For the purposes of test, all ROSS data items should be populated. 1 Data_Migration_Plan(ga510902)GA doc

4 2.2 Data Source The source for MIRPS data to be used in this effort is the USDA Forest Service (Region 5) which developed and maintains MIRPS through a cooperative effort with the California Department of Forestry and Fire Protection. MIRPS correlates with ROSS at just better than 50%. The implications of this are: ROSS data not in MIRPS will have to be hand entered to complete the MIRPS records in ROSS, and the underlying data in MIRPS covers a limited geographic area of the country, so other data sources for ROSS must be found to compensate for the limited scope of the data in the MIRPS database. 2.3 Initialization Method(s) The methods will be export and import for direct corollary tables and ASCII delimited dump with subsequent SQL*Loader invocation for more complex data sets, or the use of specially developed SQL scripts. Some data may require manual input from a workstation. 2.4 Specific Support Software Requirements No software outside that provided by Oracle will be required for this migration. However, use of tools such as those provided by Informatica could substantially reduce the time required to migrate data. Use of such tools would standardize the approach and make the process both repeatable and reliable. 2.5 Data Validation Data validation is covered in a separate document but referenced here in paragraph 3.1. The Validation effort is a relatively simple one and will rely on valid (hygienic) data being supplied by the customer. 2.6 Planning During the next phase of the ROSS project, MIRPS data will be imported using the techniques described in this document. Storage will be set aside to duplicate the size of the anticipated operational database with a 20 % margin to accommodate formal testing requirements. A development database without the margin will be created for the purpose of build and unit testing. Currently held MIRPS data may be replaced with more current data as the time to load these databases approaches. Although the procurement of MIRPS data is a customer responsibility, its loading is the task of the ROSS development team. It is estimated that the loading of the MIRPS data will consume up to 160 labor hours from either the database development staff or a DBA Tasks Completed The ROSS Data Items Requiring Value Initialization (2.1) and Data Source (2.2) selection tasks are effectively complete within this document. The determination of Initialization Method (2.3) for use in migrations is also complete. As noted above, in paragraph 2.4, no specific software will be required in support of MIRPS to ROSS migration Initialization Method (2.3) The application of specific methods to specific pairings of MIRPS and ROSS tables remains to be completed. This effort is driven by the need points in the ROSS development and testing 2 Data_Migration_Plan(ga510902)GA doc

5 schedules. ROSS tables will be populated in dependency order with look up tables being populated first and those dependent upon them by reason of foreign keys being populated subsequently. These tasks should occur when both the ROSS and MIRPS database baselines are stable Database Validation Database validation will occur concurrently with migration efforts. Ultimately it assures reliable database contents for build and subsequent deployment. This effort will involve developers, customer subject matter experts and participants representing the systems providing data for migration. Validation will in general be done on the entire ROSS database, not just that portion being migrated from MIRPS. This effort will be planned separately, but is intricately related to the migration effort Execution of Migration Execution of the MIRPS migration will be accomplished early in phase 4 using the resources called out in 2.6 above. 3. Development Environment Database Migration 3.1 Development Database Validation The validation activity is directed at determining the proper value for each controlled data item in the development database, documenting that value, and documenting the method by which the value was determined. A validation report will be prepared containing: 1. Data item identification 2. The table(s) in which it resides. 3. Acceptable filed formats and representative values. 4. Source - the source of the value. 5. Algorithm - if the value is determined by a calculation, then the calculation will be documented as well as the source of the parameters that are a part of that calculation. If the data is the result of an analysis, then that will be documented either directly or by reference in the report. 6. Data interdependencies - for each parameter, an identification of any other parameters in the database whose values are dependent on its value. 7. Units if the value is quantitative, then the units of measure which it represents must be declared. 8. Data significance, e.g., data must consist of six significant digits 9. Comments - any further explanation that may help to support the above information. 10. Responsibility - the organization that is accepting responsibility for the accuracy of the control value. 3 Data_Migration_Plan(ga510902)GA doc

6 11. Data quality - in the event that there are data values for which the proper source is unavailable in time for delivery, this shall be noted in the report and brought to the attention of the customer. Some parameters, such as a weighting factor in an optimization algorithm, may require a mathematical analysis; others may represent measurable physical parameters, such as the weight of an aircraft, and may require that the measurement be made and documented. In some cases, values may be provided by, or calculated from values provided by, a customer or another contractor who accepts responsibility for its accuracy. This might be provided either in a computer-readable form or in a document such as an interface control document (ICD). In these cases, validation may consist of determining the relationship between the parameter received and the parameter in the database. Where the data has been provided in computer-readable form and is loaded by a program, the validation will include the checking of the loading program. In some cases, test procedures may require operational values in the test database(s). In that situation, validation, migration, and verification will be performed to assure the delivery of quality data. Prior to performing any tests that require validated/verified data, validation/verification activities will be performed, and a report documenting the results of those activities will be presented at the Test Readiness Review. There will be a Database Validation Report that will contain, either directly or by reference, the field definition for each controlled data field and the rationale to support the chosen format, if appropriate. 3.2 Development Database Migration This task is composed of the effort necessary to migrate and/or to manually enter the deliverable data values into the development database as specified by the MIRPS to ROSS database validation report. The data will be manipulated coincident with ROSS module development and unit level and integration testing. The database migration effort may require manual entry and the use of Oracle Import/Export techniques, such as the use of SQL*Loader and/or the use of specially written SQL scripts. 3.3 Operational Database Verification The database verification activity is directed at ensuring that the contents of the migrated data are in agreement with the values as defined by the validation activity. This may be done by generating a listing of the file(s) being verified using either a system utility or a special purpose listing program that has been thoroughly tested, and then manually comparing the results with the validation report or the source data. Wherever feasible, this should be automated by running a program that compares the data values and/or format with those stored in the repository as a result of the validation effort. Once all of the data in a file or table has been verified, it will be saved for archival purposes. The results of this effort will be documented in a database verification report, the relevant portions having been presented at the respective test readiness reviews. 4 Data_Migration_Plan(ga510902)GA doc

7 Appendix A. Mapping of MIRPS Data to ROSS A-1 Data_Migration_Plan(