EO Data Portal Case Studies

Transcription

1 EO Data Portal Case Studies Serco Richard Campbell Copernicus Climate Data Store Workshop 3 6 March

2 Delivering Essential Services Worldwide Europe Africa, Middle East & Asia The Americas Sales 6bn Profit 300m Order Book 20bn 60% 18% 22% Over 140,000 employees 50 countries 700 ctracts 420 Space staff in UK 370 Space staff in Europe 500 Space staff in US

3 EO Capability Sentinel PDGS Technical Support Sensor Performance and Product Algorithms Ground Segment Engineering Scientific and Technical Support to EO Exploitati Ground Segment Operatis Data Processing and management

4 Sentinel-1 Scientific Data Hub users Rolling Archive ~40000 products 400+ TB of data downloaded by the users Maximum of 2 ccurrent downloads per user Ref: 1/missi-status

5 DHuS Core Main Features Online Access to EO Satellites products Optimised and Scalable architecture for Big EO Data management and bulk disseminati Optimised DB design and access to data Optmised for managing a huge number of ccurrent users requests User supporting different scenarios: Open and Free access via self registrati Restricted access Users Quota (e.g. maximum number of ccurrent downloads per user) Datasets Access Public collectis management Restricted collecti management EO Products Search, Preview, Inspecti and Download Intuitive Web and Scripting Interface for bulk download via http open data (Odata) protocol Customisable Statistics and reporting modules

6 EO Portal Studies - Data Service Initiative Data 6

7 CFI EO DataSet + Processor CFI EO DataSet + Processor CFI EO DataSet + Processor Data Service Initiative in e Slide the need for data management Collecti Collecti Collecti Validati Readiness Review Roll-out Validati Readiness Review Roll-out Validati Readiness Review Roll-out DSI Service Csolidati Csolidati -> ESA ctributes process and product requirements not solutis / technical CFIs Validati Readiness Review Roll-out Validati Readiness Review Roll-out Integrati -> Serco as Prime has the full technical respsibility Integrati -> financial aspects managed through a cost model for costing of new projects based already settled parameters -> overall competence includes IT skills, EO data management skills, service management skills Validati Validati Processing Processing Validati Readiness Review Roll-out Validati Readiness Review Roll-out Deliverable Repatriated EO DataSet Deliverable Repatriated EO DataSet Deliverable Repatriated EO DataSet Data Requirements Maintain cfigurati informati Cis at product level Ccurrent project capability Visibility of projects at all stages Mechanisms to manage change to data Traceability and history of all activities Ability to distinguish between different product groups of the same type (i.e. Master data set) Manage assets associated with the project

8 Project reporting and Mitoring 8

9 Data Cfigurati and Change Requirements Broad requirements fall largely into 2 categories Answer operatial questis Which dataset has been used in the last reprocessing campaign for a specific missi/instrument? Which IPF has been used to generate this set of data? Which characteristics? Which auxiliary? What documents rare elated to?... How many failures have been encountered during last reprocessing campaign? Support managerial decisi making How many entities/users requested a dataset? does it warrant a project? What type of data is most in demand by the community? Which is the best dataset to use for further investigati/ science activities?

10 Operatial decisi Support 10

11 Data - presentati layer 11

12 DSI - Data Informati System Serco s Soluti Integrated to support data Cfigurati as well as data change management Data Model based Data Items Dataset Product File Transformati Change Request Processor/tool Input Data Attributes Missi/Instruments/Temporal and Geographical coverage Quality Origin Product Dependency Other (e.g. volume, cloud cover ) 12

13 Populati of DSI Data Informati System XMLs Availability notificati 1 GETXML 2 Facility ( ) Internet Ingesti Tool (Service Support Desk) 3 XMLs Transfer (FTP/SSH) Update 4 XML generati for each phase of the processing: Collecti Csolidati Processing Repatriati CM DB SERCO 13

14 EO Portal Studies Cclusis 14

15 Lesss learnt The informati retained is ly as good as the underlying data model Attenti needs to be given to making the relevant data available to users in their required format Ingesti still needs specific verificati and adapti The system requires Clarity and definiti of interfaces, processes and workflows 15

16 Benefits and Value of this approach Ctrol of key asset and their attributes : Datasets Better data management and ctrol Better identified / higher quality inputs to scientific activities Allows data to be archived in an known state more intensive use of the data Facilitates simulati and multi-versi dataset management Ease of which data can be amended Areas for change are easily identified and ring-fenced Creati of logical datasets i.e. for Geography or sensor characteristic etc. 16

17 Questis? (and Answers ) Thanks for you attenti [email protected] 17