GSICS Archiving Strategy GSICS Data Preservation Strategy. Peter Miu / Boulder, USA

Size: px

Start display at page:

Download "GSICS Archiving Strategy GSICS Data Preservation Strategy. Peter Miu / Boulder, USA"

Magdalene Wilcox
8 years ago
Views:

1 GSICS Archiving Strategy GSICS Data Preservation Strategy Peter Miu / Boulder, USA CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET, USGS, WMO 1

2 Overview Clarification: Archive vs Preservation Identifying what GSICS Products to preserve. Data Preservation Strategy. Requirements and Summary. 2

3 Data Archiving vs Data Preservation The term data archiving not only implies storage but additional services such as: data discovery services for users to detect the data (making the data explorable from multiple sources). data identification services for users to clearly understand what the data is (e.g. meta-data, landing pages, etc.). data search services for users to find the data. data migration obsolescence mitigation services to ensure the components of the archive maintain a reasonably level of integrity e.g. media change. These services make data archiving expensive. Data preservation is less of a formal endeavour to ensure that digital information of continuing value remains accessible and usable. It only imply reliable access to the data. For GSICS products, a data preservation strategy shall apply. 3

data migration obsolescence mitigation services to ensure the components of the archive maintain a reasonably level of integrity e.g. media change. These services make data archiving expensive.

4 Identifying what to Preserve Do we need to preserve everything? 4

5 GEOLEOIR NRTC Products Analysis EUMETSAT GEOLEOIR Near Real Time Correction (NRTC) Product Analysis: One product has one correction applicable to support real time product generation. Products have a shelf life after which time they are no longer useful. For an ATBD, EUMETSAT can regenerate all NRTC products if needed. Do we need to preserve NRTC GSICS products? Recommendation: No. 5

Products have a shelf life after which time they are no longer useful.

6 GEOLEOIR RAC Products Analysis EUMETSAT GEOLEOIR Re-Analysis Corrections (RAC) Product Analysis: RAC is a combined product replaced daily with a new correction. RAC products are expected to be available indefinitely i.e. no shelf life. A RAC product will continue to grow until an ATBD update. A new MAJOR VERSION of the RAC combined product file is then created for the same time period as the version it replaces. The old version of the RAC product is moved but still accessible for reference. All versions of a RAC product should be available for reference. EUMETSAT can recreate any ATBD version of a RAC product if needed. Do we need to preserve RAC GSIC products? Recommendation: Yes. 6

A new MAJOR VERSION of the RAC combined product file is then created for the same time period as the version it replaces.

7 Data Preservation Strategy (1) For the latest RAC products version, product regeneration is done through: By design as a RAC product is generated daily with a new correction which replaces the existing RAC file. In addition to this, RAC products are expected to be replicated : Across all collaboration servers. Through System administration of the servers where cluster technology is used to ensure service always available and high availability disk arrays. In addition to this, backups of disk are periodically taken to simplify data recovery. Worst case scenario is the RAC product can be regenerated by the product generation software. 7

Through System administration of the servers where cluster technology is used to ensure service always available and high availability disk arrays.

8 Data Preservation Strategy (2) For the Old RAC products versions, product regeneration is done through the replication as described for the latest version: Resulting from the mirroring across all collaboration servers. Through System administration of the servers where cluster technology is used to ensure service always available and high availability disk arrays. In addition to this, backups of disk are periodically taken to simplify data recovery. Worst case scenario is the RAC product of any version is regenerated by the product generation software. 8

9 Requirements and Summary To implement GSICS Product Preservation, the following Data Management Systems are required to be in place: Mirroring on the collaboration servers for all GSICS products. Collaboration Server Administrators need to ensure service and backup strategies are in place. GSICS products generation software should be developed to be able to regenerate an version of a GSICS product. The last point is a mandatory requirement in the development of the GSICS product generation software. 9

Collaboration Server Administrators need to ensure service and backup strategies are in place.

10 End of Presentation: Thank you for your attention 10

Calibration Event Logging Status and Plans

Calibration Event Logging Status and Plans Presented to: CGMS-43 WGII/8 Paper: CGMS-43-EUMETSAT-WP-10 Outline Introduction Task Team on Calibration Event Logging Status WMO-OSCAR landing pages Towards