Copernicus Space Component ESA Data Access Overview J. Martin (ESA), R. Knowelden (Airbus D&S)
Introduction and Context Operational Scenarios Translation into interfaces Translation into services Current Data Access Landscape Introduction to Evolutions Feedback Process
The CSCDA System Introduction The Copernicus Space Component Data Access (CSCDA) System provides comprehensive and coordinated access to EO data products from multiple satellites to Copernicus space data users across Europe. It allows the capacity: To link transparently the different EO Data Providers and the various Copernicus Service Projects (CSPs) using coordinating functions. To create synergy and sustainability across the various Copernicus Contributing Missions (CCMs). To facilitate the data access for the services and aim at long term data preservation and access. The CSCDA manages the coordinated production of Data Sets: multimission coherent data collections pre defined according to specific user needs in terms of data type and mode of operation. They are produced under pre agreed schedules or as an ad hoc response to user requests. A key component of the CSCDA is the Coordinated Data Access System (CDS), ensuring Data Set construction and dissemination by coordinating the CCM s production and dissemination flows.
Data Flows from CCMs to Copernicus Users
End to End Operational Scenarios CSCDA Scenario SCE 01 Data Streams from Global Systematic Missions REPROCESSING ARCHIVE RETRIEVAL TASKING SCE 02 SCE 03 SCE 04 SCE 05 SCE 06 SCE 07 SCE 08 SCE 09 SCE 10 SCE 11 SCE 12 SCE 13 SCE 14 Regional Monitoring (direct flow) Regional Monitoring (via CDS) Geographic Area Coverage Background Acquisition Rush Satellite Tasking Crisis Monitoring Rush Retrieval from CCM Archives Normal Retrieval from CCM Archives Direct Retrieval from Archives of Global Systematic Missions Consolidated Data Streams from Global Systematic Missions Reprocessing of Historical Series for Global Systematic Missions Direct Retrieval from CDS Coordinated Archive Off Line Mapping in Non Rush Mode Predefined On Demand Emergency
Scenarios Demonstration Demonstrations are provided of two scenarios. These provide a good overview of CSCDA functionality and interfaces, and the tools used, showing both future tasking and archive retrieval in an emergency context. 1) Scenario 6: Rush Satellite Tasking 2) Scenario 8: Rush Retrieval from CCM Archives Both scenarios are triggered by requests from an authorized user for archived and tasked data, which are received and processed by a CDS/REACT (Rapid Emergency Activation for Copernicus Tasking) Operator. The scenario uses as its basis a hypothetical flooding event in central Europe. NB: only the principal steps are shown.
Scenario Initiation Emergency Service REACT Operator CCM Flooding Occurs User requests satellite data by completing and submitting online SPERF forms Reception of SPERF forms. Creation of sub data sets. Possible missions selected. Propagation to CDS subsystems.
SCE 06: Rush Satellite Tasking Emergency Service REACT Operator CCM GEST tool used to create emergency dossier and identify candidate CCM(s) Emergency Order forms sent to selected CCMs Optional check with user Feasibility check and final confirmation Order Status monitoring Retrieval of Data Product Order receipt and acknowledgement. Feasibility analysis. Mission planning, acquisition, processing Data dissemination on ftp pick up point Catalogue, archiving. Placement of data in Emergency Dossier Circulation to CDS
SCE 08: Rush Retrieval from CCM Archives Emergency Service REACT Operator CCM GEST tool used to create emergency dossier and search for candidate archived products via HMA Catalogue Interfaces HMA Emergency Order sent via HMA Ordering Interfaces Order Status monitoring Retrieval of Data Product Order receipt. Product retrieval & processing (if required) Data dissemination on ftp pick up point Catalogue & Archiving. Placement of data in Emergency Dossier Circulation to CDS
From Scenarios to CDS Interfaces Definition of Interface Standards towards CCMs Favouring re use of HMA / OGC and other standards
From Scenarios to a Service Charter The CSC Ground Segment provides a set of main services to all users, independently from the access infrastructure + Research Support Service and Hosted Processing Service: Support prototyping of new algorithms on a dedicated platform new service under definition 19/03/2013 11
Current Copernicus Space Data Access Landscape
Open Access The Scientific Data Hub Initial Operations: Self Registration and Sample Products open since S 1A launch 04 2014 Routine Data flow opened following IOCR 10 2014 Rolling Archive Initial 12 months data population to support Satellite Ramp Up phase no timeliness guarantee Quota restriction of 2 concurrent downloads to ensure bandwidth availability for all users Go to http://scihub.esa.int
Hub Access A common solution for Open / Collaborative/ International Simple UI Powerful API Cross platform and compatible with many programming styles Script, Java, App RESTFul / HTTP Simple Authorisation OpenSearch OpenData
Sentinel 1 User and Data Statistics (open access data hub) By 26 March 2015: 5979 registered users 77536 products available for download 543211 products downloaded by users, representing about 680 TB of data
Roadmap for Data Hub New user interface under development to better support multi Sentinel context S 2 support in line with satellite commissioning S 3 support in line with satellite commissioning S 5P support in line with satellite commissioning Open Source software release via public repository in June 2015
Current Copernicus Space Data Access Landscape
Dedicated Data Access for Copernicus Dedicated services for eligible users of Data Warehouse Datasets Point of entry via https://copernicusdata.esa.int Subject of major evolutions in 2015
How to access the data today? FTP for large volumes (with group account) Copernicus Client (Eoli SA) for single products (with user account)
Introduction to Evolutions New CCMs/CCMEs DataWareHouse 2.0 Requirements Parallel/Related Technology Developments for CDS Continuity with current baseline CDS v2.1 CDS v3.x INSPIRE Feedback from Routine Operations & End Users HMA Interfaces Extension Additional requirements captured during the engineering reviews 20
Data Access Interface Evolutions Core data access services offered through a uniform webbased interface, supported on major browsers Comprehensive DISCOVERY, VIEW and DOWNLOAD services provided towards full archive Friendly to Web and OGC standards for Earth Observation Detailed product image as map layers (in selected cases) Sharable shopcarts, dataset search, browse and product URIs Systematic user defined downloads using performant download manager Progressive deployment for S 1, 2, 3 and CDS from Q2 2015
Data Access Interface Evolutions
Feedback processes To helpdesk: eosupport@copernicus. esa.int To Account Managers, copernicus.account.ma nager@esa.int via feedback in questionnaire Via workshops
Thanks for your attention! User REGISTRATION HELP DESK Service User ACCESS MANAGEMENT OFFLINE MAPPING Service Coordinated Access Continual Improvement DISCOVERY Service EMERGENCY Service VIEW Service BULK DISSEMINATION Service DOWNLOAD Services