Sentinel Collaborative Ground Segment Sweden Final Report

Transcription

1 Sentinel Collaborative Ground Segment Sweden Final Report Prepared for the Swedish National Space Board Document ID Version SM-CGSS-FREP Change record Version Date Author Final delivery to SNSB L Edgard, I Spence, D Åsvärn Spacemetric AB Tingsvägen Sollentuna Sweden

2 0. Executive summary In March 2014, ESA presented its plans to the Swedish National Space Board (SNSB) regarding the future Swedish user access to Sentinel data. Subsequent discussions led to SNSB entrusting Metria and Spacemetric with the task to investigate and report upon Swedish needs for Sentinel data, and the technical solutions necessitated by these needs. Broadly speaking, Metria was given the brief to investigate the needs, whilst Spacemetric was asked to shed light on the more technical aspects. This document is the result of Spacemetric s work, focusing on technical issues regarding the different alternatives for how Sweden, in a cost-efficient way, could access, archive, catalogue and disseminate data from the Sentinel satellites for Sweden s national needs. It has in some cases been found to be difficult to obtain a clear picture of the Sentinel system capabilities and constraints. In consequence, some of the findings and recommendations are based on verbal communication with ESA officials only. As a result, the recommendations offered must be taken with a grain of salt, necessitating later confirmation as the Copernicus system matures. Most importantly, the study has identified a clear need for a Collaborative Ground Segment for Sweden (CGSS). Additionally, it has identified several overall requirements to be placed upon this new facility: Responsibility for archiving and curation of Copernicus data relevant for Swedish users Enabling timely access to Copernicus data over Sweden s Immediate Area of Interest Providing Copernicus products and services adapted to Swedish use (e.g. map projections) Gathering Copernicus data over other Areas of Interest for Sweden/Swedish users Need for creation of clear and open principles of governance. Together these requirements form a clear scope for the CGSS. In order to meet this challenge the following recommendations are offered from the study: i. Sentinel data should be continuously downloaded at a Swedish hub/mirror site, giving easy access to recent and historic data ii. Make further analysis of the needs, technical modalities, and costs associated with organising access to direct-download data from the Sentinel satellites iii. Provide both a Swedish archive of relevant ESA products as well as processing functionality for specifically Swedish Sentinel products (i.e. use Swedish map projections and elevation data) iv. Consider integrating other datasets with similar characteristics to the Sentinels (e.g. Landsat 8) v. Make an analysis of ESA reporting requirements upon the CGSS as an input to the ongoing negotiations with ESA regarding a Swedish Collaborative Ground Segment agreement vi. The Swedish National Space Board should lead the work to define the roles and responsibilities of a governing body for CGSS including options for its potential form (e.g. government agency, public/industry consortium). The most immediate action required is to make contacts with ESA soonest with a view to securing access all relevant Sentinel-1 data received to date. In terms of the establishment of the CGSS, this report summarises a two-phase approach. The initial establishment of the CGSS lasts 12 months and is then followed by ongoing operations and maintenance. The total cost for the establishment is estimated at MSEK with ongoing operations and maintenance estimated at MSEK per year. 2 SM-CGSS-FREP-10

3 Table of Contents 0. Executive summary 2 1. Introduction 4 2. Definitions, acronyms and abbreviations 4 3. Reference documents 4 4. Background Background Established Swedish activities Acknowledgements 5 5. Collaborative Ground Segment Context Operational scenarios of Sentinels Sentinel data access channels 6 6. Technical solutions Background Rely on user data access directly from the ESA data hub, or download data to a Swedish hub for user data access from here? Is there a need for direct delivery of Sentinel data through receiving stations in Sweden or in neighbouring countries? Data volume and retrieval aspects Products Reference systems The role of the Saccess archive Supplementing the Swedish part of the collaborative ground segment with other types of data Licensing Building blocks of the Swedish Collaborative Ground Segment Operational and maintenance aspects Steps towards a working CGSS Cost assessments Background Scenario System cost components 24 3 SM-CGSS-FREP-10

4 1. Introduction This document presents different alternatives for how Sweden, in a cost-efficient way, could access, archive, catalogue and disseminate data from the Sentinel satellites for Sweden s national needs. Broadly speaking, the document focuses on issues of a technical nature, i.e.: Technical solutions Reference systems The role of the Saccess archive Operational and maintenance aspects Cost assessments Licensing The texts are intended to be possible to use as a basis for decisions as to how the Swedish part of the collaborative ground segment for Sentinel data shall be organized. 2. Definitions, acronyms and abbreviations API CGS CGSS CSC DEM DHuS EO ESA GCP GMES GO MS NGO ROM SNSB TB Application Programming Interface Collaborative Ground Segment Collaborative Ground Segment Sweden Copernicus Space Component Digital Elevation Model Data Hub System Earth Observation European Space Agency Ground Control Point Global Monitoring for Environment and Security Governmental organisations Member State Non-governmental organisation Rough Order of Magnitude Swedish National Space Board Terabyte 3. Reference documents [RD-1] Sentinel Overview and Status, Meeting with Swedish Partners, 18 March 2014 [RD-2] Copernicus Space Component Collaborative Data Access Data Hub - Mirror Site Interfaces, J. Martin, European Space Agency, 18 March 2014 [RD-3] Dnr 180/14, SNSB Background 4.1 Background During a Swedish National Space Board (SNSB) meeting in April 2014, ESA presented its plans regarding the future Swedish user access to Sentinel data. Subsequent discussions, i.a. by the stakeholders of the Swedish Saccess database, led to the SNSB in May 2014 producing a document ([RD-3]) describing the need for investigations regarding future the Swedish needs for Sentinel data 4 SM-CGSS-FREP-10

5 and the technical solutions necessitated by these needs. Metria and Spacemetric were entrusted the task to produce a report regarding these issues, i.e. future Swedish needs for Sentinel data and the technical solutions necessitated by these needs. Broadly speaking, Metria was given the brief to investigate the needs, whilst Spacemetric was asked to shed light on the more technical aspects. This document is Spacemetric s contribution, focussing on the technical issues of what is referred to as the Collaborative Ground Segment Sweden (CGSS). It has in some cases been found to be difficult to obtain a clear picture of the Sentinel satellite system capabilities and constraints. In consequence, some of the findings and recommendations are based on verbal communication with ESA officials rather than on official documentation. As a result, any recommendations offered must be taken with a grain of salt, necessitating later confirmation of some of the information. 4.2 Established Swedish activities Sweden has a long tradition in satellite remote sensing, reaching back to the 1970s through the reception of early Landsat data. This tradition was strengthened and extended through active participation in the Spot programme from the early 1980s. In parallel, there is a long-standing interest in environmental and forestry remote sensing applications, notably in the forest sector, where there is a high level of maturity in services for annual follow-up. The Saccess initiative became operational in 2008, offering public access (with specific licensing requirements) to the data primarily collected for these purposes. The Sentinels, and particularly Sentinel-2, offer the opportunity to move from an annual cycle into the ongoing monitoring of phenomena, building upon already established services in Sweden as well as creating new possibilities. 4.3 Acknowledgements Spacemetric would like to extend its thanks to the following individuals who made themselves available to the study team during the course of the project: Bianca Hörsch Jolyon Martin Jyri Heilimo Olli-Pekka Mattila European Space Agency European Space Agency Finnish Meteorological Institute Finnish Environment Institute (SYKE) 5. Collaborative Ground Segment Context 5.1 Operational scenarios of Sentinels The Sentinels have distinct sensing capabilities and associated operational concepts (discussed in more detail in Metria s companion report this one) that form a backdrop to the technical aspects of CGSS study discussed in this report: Sentinel-1: is a radar mission with acquisitions planned over specific regions for periods of several months, the regions selected changing regularly according to a high-level data plan Sentinel-2: is an always on high-resolution optical sensor operating over land areas Sentinel-3: is a suite of always on sensors for synoptic data supporting oceanographic, land and atmospheric monitoring. From these summaries it is clear that Sentinel-1 s campaign-based data acquisition means there is no guaranteed continuity of coverage of any given area during the mission. This should, however, not be 5 SM-CGSS-FREP-10

6 the case for Sentinel-2 and Sentinel-3. Whilst Sentinel-2 data has similarities to data currently used for land applications in Sweden, Sentinel- 3 data closely resembles data already used for meteorological and hydrological applications in Sweden. Thus it should be relatively simple to integrate Sentinel data into existing Swedish activities, given easy data access. 5.2 Sentinel data access channels The different channels for access to Sentinel data are outlined in [RD-1] and together form the Copernicus Space Component Ground Segment Data Access (Figure 1). The channel(s) available to any given user depend upon their category. Figure 1 CSC Ground Segment Data Access (from [RD-1]) Copernicus Services Access Copernicus Services have special access to Sentinel data including full on-demand access to the archive and the ability to request emergency programming of the Sentinel satellites. This is effectively VIP data access and is superior to any of the access categories. Membership of Copernicus Services is rather exclusive and has become established through the formation of consortia over the many years of the GMES/Copernicus programme Scientific Other/Access The Scientific/other users have free and open access to a rolling archive (i.e. a few months 1 ) of data from the Sentinel satellites. This access is however, shared with all users of this category, spread around the globe, who as a consequence are likely to be large in number. For this reason it is planned that there will be some sort of (as far as we know as-yet undefined) quota management to guarantee some level of data access to all users. Beyond the rolling archive no further access is planned for this group (i.e. no access to archived data) International Access Agreements The International Agreements Access is in order to honour pre-existing agreements and will be achieved via a dedicated network. The data access will be to a rolling archive which will allow the 1 According to verbal information, this period might be as short as two months. 2 According to verbal information, this period might be as short as two months. 6 SM-CGSS-FREP-10

7 construction of mirror archives elsewhere in the world. There is no access provided to the archive of older data held by ESA Member States Collaborative Access The Member States Collaborative Access will be via the ESA Data Hub System (DHuS) through a rolling archive of a few months data 2 with guaranteed access for the cooperating Member States (MSs). There will be only a limited online access to archived data, supplemented by the possibility for requests for offline bulk delivery on a campaign basis. It is planned that the more than 20 cooperating MSs will share a single line currently specified at 3 Gbps 3. A programmatic interface will be made available for systematic data search & download (i.e. OpenData, later OpenSearch) while a subscription interface may be made available at a later date Current access status The Sentinel-1 mission has achieved in-orbit acceptance and the ground segment is now under a ramp-up phase. Initial data is already being made available through the ESA Data Hub under the scientific-user access mechanism (scihub.esa.int). 5.3 Preconditions for the Swedish ground segment The ESA Data Hub System is the collection point where Sentinel data will be available for download by Member States through the ESA Member States Collaborative Access. The idea is to offer each Member State a single access point for Sentinel Data. Each Member State will be issued with a single username and password with one entity in each country coordinating data access. This implies that each of the Member States will have to implement a download scheme or harvest pattern to copy the relevant data to a national data hub from where users access the Sentinel data. This national data hub becomes a mirror, complete or partial, of the ESA data hub. A national mirror site can be a full copy of the ESA data hub or just contain a geographical selection. A national mirror can host the data forever as a long-term archive, or act as a rolling archive. It is up to the national entity to decide how to implement the national mirror site. However, the national mirror will be the only way for public and industry in a specific country to access free Sentinel image data, at least in a timely manner and in large amounts. SWEDEN ESA National mirror ESA MSs Collaborative Access CGSS Collaborative Ground Segment Sweden Internet 3 GBit/s DHuS ESA Data Hup System Long term archive xx years of data Bulk dissemination Rolling archive a few month of data Figure 2 CGSS access to the ESA DHuS 2 According to verbal information, this period might be as short as two months. 3 ESA has indicated verbally a preparedness to expand this to 10 Gbps as the need arises. 7 SM-CGSS-FREP-10

8 France and Germany have concrete plans to harvest all data available in the Data Hub. That means global coverage. The reason for this is to guarantee German and French public and industry users free and open access to a global dataset. The single shared connection to the ESA Data Hub is a limiting factor. A Swedish mirror has to compete for resources making bulk download. 6. Technical solutions 6.1 Background In Spacemetric s brief, it is stated that the report should suggest alternative technical solutions (both software and hardware) as well as the volume/dimensional aspects of these solutions. Also, aspects of how information could be retrieved from the database should be described. Below we discuss a number of important themes that are relevant to this task. 6.2 Rely on user data access directly from the ESA data hub, or download data to a Swedish hub for user data access from here? Scientific as well as other users are planned to have free and open access 4 to a rolling archive (i.e. the last few months) of data from the Sentinel satellites. This access is however, planned to be shared with all users of this category, in all countries. The number of users is, as a consequence, likely to be large. For this reason, there are plans for an (as far as known yet undefined) quota management system in order to guarantee some level of data access to all users. There is, however, at the moment of writing no way of knowing whether or not the supply of recent imagery from ESA hubs will be sufficient to keep up with the data requests from prospective Swedish users. In addition to the possibilities for any individual user to download recent data (i.e. up to a few months old), the Member States (MSs) will have a so-called Collaborative Access via the Data Hub System (DHuS). This will be done through a separate rolling archive with guaranteed access for the cooperating Member States. There will, however, be no immediate access to older archived data other than via requests for offline bulk delivery on a campaign basis. Any Swedish user need or application necessitating longer time perspectives than available in the rolling archive would therefore require that data be requested from ESA. Alternatively, the Sentinel data would need to be continuously downloaded at a Swedish hub to facilitate access to historic data. The simple fact that data will be accessible means that this alternative is expected to greatly increase the likelihood of use by Swedish users of historic data. To sum up, if Sweden wants to ensure easy and reliable user access to the rolling archive (i.e. recent data), it appears that the Sentinel data would need to be continuously downloaded at a Swedish hub ( mirror site ) for easier access. Should an even higher degree of speed be deemed necessary, downloading the data directly from the satellite, to a Swedish antenna or to an antenna of a neighbouring country, could be a complementary solution. As for historic data (i.e. older than the rolling archive), a Swedish hub is the only practical way to 4 This is understood to mean free and open data access, i.e. open to any user, regardless of domicile. 8 SM-CGSS-FREP-10

9 provide reliable access for Swedish users. This is yet another strong argument for recommending the creation of a Swedish hub (mirror site) Recommendation Sentinel data should be continuously downloaded at a hub/mirror site, giving easy access to recent and historic data. This hub should have a system allowing a user to find and access data. It should be possible to do this with a minimum of previous knowledge and experience. Furthermore, as an immediate measure, since data from Sentinel-1 is reportedly already, at the time of writing, being loaded into the ESA hub, it is recommended that contacts be made with ESA soonest with a view to downloading all relevant Sentinel-1 data received to date. 6.3 Is there a need for direct delivery of Sentinel data through receiving stations in Sweden or in neighbouring countries? Information from ESA points to three hours being a probable time delay from data reception to its delivery to the ESA hub when it comes to data over Europe, i.e. all areas included in the main Swedish area of interest. For areas outside Europe, this latency period is reported to be 24 hours. To these delays should, however, be added the time needed for delivery from the ESA hub to Swedish users and/or a prospective Swedish mirror site. The Metria investigators has found that there are some potential Swedish Sentinel data users with greater needs for a speedier data delivery than offered via the ESA hub. Two examples of such users are meteorologists and emergency services users. The needs of such users would present a strong rationale for contemplating more rapid delivery methods than offered via the ESA hub, i.e. downloading of data directly from the Sentinel satellites. However, although the team of investigators has been able to identify a need for speedy data delivery from the Sentinel satellites to some Swedish users, it has been found difficult to quantify the benefits associated with a speedier delivery. Direct downloading of data from the Sentinel satellites could increase the speed with which Swedish users could access the data. However, according to the information made available to the team of investigators, the potential benefit would differ somewhat between the different satellites. Sentinel-1 and 3 appear to promise a more likely direct delivery for Sweden, when a Sentinel satellite passes in the vicinity of the main station in the area (in this case Svalbard), the satellite acquires and downloads imagery of an area encompassing Sweden and its immediate AOR (cf. the report from Metria for a description of this area). This means that a Swedish receiving station, or a station in a neighbouring country, would make it possible to download imagery over the Swedish area of interest 5. However, Sentinel 2, when passing Sweden and its neighbouring countries, is apparently likely to then download recorded data mainly from other parts of the world. This decreases the likelihood of the Sentinel 2 imagery received by a Swedish receiving station (or in a neighbouring country) actually being over the area where the satellite passes. Also, it is perhaps important to point out that there is probably no guarantee for the success of direct downloading - in any given situation, ESA retains control over the satellite, and there is thus no guarantee for that a specific satellite passage will result in a particular acquisition being downloaded. However, regardless of the differences in modus operandi between the different Sentinel satellites, the likelihood of getting substantial amounts of data more rapidly over the Swedish area of interest must be said to increase significantly, should the data be downloaded directly from the satellite. It should be pointed out, however, that a choice to use direct downloading in no way rules out data delivery from the ESA hub to a Swedish mirror site Recommendation 5 C.f. Metria s companion report. 9 SM-CGSS-FREP-10

10 Given that there is a cost, probably substantial, associated with organising downloading of data directly from the Sentinel satellites, the team of investigators recommends that an analysis is made of both user needs, technical modalities, and the costs associated with organising download of data directly from the Sentinel satellites. Other aspects to be weighed in include the perceived need for national control of the downloading process and the deemed probability of the alternative possibility to create bilateral or multilateral co-operation agreements with data receiving stations in other countries. The weighting of the needs of time-sensitive functions for the public good should influence the decision whether or not to download data nationally (in addition to downloading data from the ESA hub to a Swedish mirror site), or to seek access to near real-time data through bilateral or multilateral co-operation agreements. 6.4 Data volume and retrieval aspects According to the information available to the investigation team, the total volume of daily acquisitions for the Sentinel Series A satellites (i.e. S1, 2 and 3) is c. 4 TB, or c. 1.5 PB per annum. For an area the size of the probable Swedish Area of Interest, as the data may be downloaded twice (for systemcorrected data and probably also for some ortho data), this would translate into somewhere in the vicinity of 50 TB per annum. Data volumes of this size are in themselves relatively easy to handle, especially if volume-efficient management and production methods are used, avoiding unnecessary data duplication. Also, the costs associated with storage of data volumes of this magnitude are not excessive. This statement would hold true even if data were to be downloaded from larger areas, such as the wohole of Europe, or the Eurostrategic area (i.e. covered by a radius of 6000 km from Brussels). Rather, even with much smaller datasets than these two areas would give rise to, the challenge will would be the complexity of the retrieval. The data will consist of many images, and there will be a great need for smart sorting and filtering. There will therefore need to be in place a system for speedy and efficient archiving and cataloguing. There will also be a need for data processing, for instance to combine images into composites and to produce value-added products specific to Swedish needs. 6.5 Products In general terms, there are two principal ways of presenting the user with satellite image products: systematic product generation on the one hand, and user-defined product generation on the other hand. Systematic product generation means that products are defined and created by the data producer. There is a focus on ready-made products, and customer interaction is limited to choosing from a smorgasbord of finished products. There is little flexibility one size, or possibly three or four sizes, fits all. Experienced users do not find much room for their ideas the output is fixed, as in most webbased services. For user-defined product generation, the focus is on customised, bespoke products. Experienced users can use the data after their own fashion the output model is not at all fixed, even if standard products can be offered alongside product solutions adapted to expert users. Typically, this solution is well suited to large data volumes, where it is sometimes better to perform processing on demand when needed, getting the correct product back as an end result. This product generation system is, for example, implemented in the Saccess system. ESA s plans for Sentinel-generated products at the moment appears to mainly emphasise systematically generated products. The Spacemetric team, on the other hand, sees a need for Sentinel data to be used in such a way as to cater for the needs of both the Swedish expert user and general public. This would imply choosing the model of user-defined product generation at the 10 SM-CGSS-FREP-10

11 Swedish hub, adapted to the expert user, with systematic product generation still being available for less expert users. Such an approach would necessitate i.a. the following functionalities: Automatic archiving of all images Generating synthesis imagery, mosaics Providing services and file-based products (WMS, WCS, FTP, http) Supporting user accounts. One advantage with the user-defined product generation approach is that there is less need to store ready-made products that tend to duplicate the same data. Instead, the data is stored only once, rapidly creating the bespoke products from this raw data by order when needed, keeping the costs for archive handling and storing down. To enable the generation of user-defined Sentinel products in Sweden, access is required to lower level products from ESA s Data Hub. These must include at least a pre-orthocorrected level (i.e. prior to Level 1C in ESA nomenclature), while the optimal product level depends upon the sensors on each Sentinel platform and the form of the final product specifications Recommendation The CGSS should archive relevant of ESA products and also provide processing functionality for both standard and customised Swedish products. 6.6 Reference systems Background In Spacemetric s brief, it is stated that the report should show how national geographic reference systems, elevation databases, and cloud masking should be applied for the data. The report should also show which solutions are already available from other entities National geographic reference systems For Swedish users, it is essential that any geodata products are available in Swedish projections (RT 90, SWEREF99, etc.). Products from the Saccess system fulfil this requirement, and in order to ensure data continuity, it is important that Sentinel data are available in the reference systems used by Saccess. None of the ESA products will meet this need, so the capability to generate such products will be necessary within the CGSS. While simple reprojection might work in some cases, other factors make this inappropriate for much of the Sentinel data. So lower level ESA products should typically be the starting point for Swedish products. Along with providing support for Swedish map systems, the Sentinel products should also be appropriately accurate within these map projections and the ESA products will not satisfy this. Products from the Saccess system fulfill these demands, but are generated manually. For a Swedish Collaborative Ground Segment, an equivalent capability to generate accurate Sentinel products must be made available, but for practical and cost reasons this should be achieved with wholly automatic methods using reference layers such as airborne imagery. This is a relevant capability for the Sentinel-1 and Sentinel-2 missions a which are relatively high resolution while for Sentinel-3 a reprojection of ESA s products may be sufficient. Since the ESA products for Sentinel data do not at the moment offer Swedish projections and are not entirely accurate relative to Swedish geography these are further arguments for the creation of a Swedish hub Elevation databases 11 SM-CGSS-FREP-10

12 Reportedly, the otherwise excellent Sentinel elevation database stretches only up to the 60th parallel. This means that there are unresolved height-related errors in the higher-level ESA products for Sentinel-1 and Sentinel-2 over much of Sweden, even if they were to suit Swedish needs in all other aspects. For this reason, for Sweden, there is a need to employ another elevation database, e.g. the Swedish National Height Database (Nationell höjddatabas, NH 6 ). This would need to be incorporated within the processing of the Swedish products for Sentinel-1 and Sentinel-2. A choice of this database would have the added advantage of the database, while not at present being available for the whole country, being more widely used in Sweden and leading to truly consistent products. Since the ESA products for Sentinel data do not at the moment include using a high-quality elevation model over the whole of Sweden, this is yet another argument for the creation of a Swedish hub Cloud masking For Sentinel-1 s radar the clouds are invisible, while for Sentinel-3 the clouds and atmosphere are an integral part of the data being gathered. But for Sentinel-2, cloud cover is an inconvenience. Information on clouds is sensed directly by specific channels on Sentinel-2, and cloud masks are generated. And while this is an asset, the issue of cloud masking is complex. Clouds are not discrete but rather there is a continuity of cloud types, and cloud shadows can be just as problematic in terms of their effect on data quality. So, despite the provision of the Sentinel-2 cloud masks, these are likely most useful only in the gross filtering of data into more cloudy and less cloudy categories. Naturally this information should be used and available within the Swedish Collaborative Ground Segment. But specific users will in many cases still want or need to be able to make their own application-specific assessment of clouds in Sentinel-2 data. Such functions could also form part of the functionality offered by the CGSS system Recommendations Swedish projections (RT 90, SWEREF 99, etc.) should be supported by the Swedish collaborative ground segment. A Swedish elevation database should be sourced for all Swedish Sentinel products. A process for automatic cloud masking should be provided with the CGSS system. The process should ideally include a method for handling cloud shadows. 6.7 The role of the Saccess archive Background In Spacemetric s brief, it is stated that the report should show if, and, if so, how, the Saccess archive and its functions could be expanded to include the Swedish part of the collaborative ground segment. If such an extension should prove to not to be feasible, the report should show how the Saccess archive and its functions could enjoy access to data from the collaborative ground segment. The report should also show how the historic data that already is in the archive should be preserved and made accessible for Swedish users and developers Saccess history and role Since 2008, a national Swedish Satellite Data Archive facility, Saccess, is providing free yearly national coverages of image data from SPOT, IRS and Landsat to any user living in Sweden, Norway, Finland or Denmark for any purpose. 6 Formerly called NNH, Ny Nationell Höjddatabas. 12 SM-CGSS-FREP-10

13 Saccess is operated by a consortium consisting of - currently - the Environmental Protection Agency, the National Space Board, the Forest Agency, SLU, Holmen, SCA, Sveaskog, Bergvik Skog, Metria and Spacemetric. Saccess contains historical datasets with optical satellite data from the 1970s, 1980s, 2000 and From 2007, Saccess is annually updated with a new nationwide dataset. The available data was acquired during the vegetation season and with a spatial resolution of metres. The exception is the data from the 1970s, which has 80-metre resolution. Through Saccess, optical data with 10 to 30- metre resolution is probably the type of satellite data that have the broadest operational use in Sweden today, the Saccess database aiming at both professional users and the general public Saccess compared to CGSS When comparing GCSS with the Saccess archive (Table 1), it is obvious that there are some important differences. Firstly, the Saccess system aims at managing a long series of co-registered images, whilst the main purpose of CGSS is to guarantee access for Swedish users to the Sentinel data stream. Secondly, Saccess is based on pre-selected, pre-processed, nearly cloud-free images available as images or aggregated into yearly mosaic. Users can download the images or parts of the mosaics. Saccess depends on operators for the selection of their images, their pre-processing with Ground Control Points (GCPs) and the preparation of image mosaics. The proposed CGSS on the other hand is a system receiving a continuous stream of images. The incoming imagery is corrected to the Swedish reference systems through an automated process, and mosaics and composites are also generated automatically based upon image metadata. Similarly to Saccess the user can download images for use with processing toolboxes, or they can use completely new services for further processing within the CGSS environment so that the downloaded data volumes are greatly reduced. The system is fully automatic, meaning there is no need for operators in the data processing, but rather for monitoring of the system and its performance only. Management of user accounts in Saccess is also a manual process that includes verification of each user s address in the Nordic region as a consequence of the current licence conditions for use of Saccess data. The user registration needs of CGSS are rather simpler, on account of the free and open data access policy, and modest reporting requirements, so this process can also be automated. The different licensing models of the two systems, Saccess and CGSS, make it impractical to combine them into a single system. In such a case, all the user handling would have to conform to the more stringent Saccess model, so CGSS would suffer a radically more complex and costly user management regime than necessary for Sentinel data. A likely consequence would also likely be a dampened uptake of Sentinel data. The licensing of the Saccess images could of course be renegotiated with the IPR owner, but probably at a significant cost. Despite the above, there is no reason why the current Saccess user conditions could not, regardless of whether or not the Saccess archive is updated in the future or not, co-exist alongside a Swedish mirror site for Sentinel data providing data access to all users, not only Nordic users. In practice, this could simply be attained by presenting the Swedish Sentinel user with a link to the Saccess portal, and vice versa. Characteristic Saccess Proposed CGSS Selection of images Manual selection for a full coverage per year Constant stream of images ingested automatically 13 SM-CGSS-FREP-10

14 Characteristic Saccess Proposed CGSS Swedish reference system Generation of products Manual co-registration to Swedish reference system Manual creation of single yearly mosaic in several flavours Automatic co-registration to Swedish reference system Automatic creation of images, mosaics and multi-day composites Access to data Image download using FTP or http Service-based access (e.g. WMS) and image download using FTP or http Support for further processing None Service-based processing and data processing toolboxes User accounts Manual validation of user accounts Automatic user account validation Licensing Strict limitations (no redistribution, only users in the Nordic countries) Open and free licence Table 1 Comparison of Saccess and CGSS primary characteristics Recommendations The Saccess archive should, at least initially, co-exist alongside a Swedish mirror site for Sentinel data that provides data access to any user. This could be attained, for instance, by presenting the Swedish Sentinel users with a link to the Saccess portal, and vice versa.. The Swedish part of the collaborative ground segment should be designed in such a way as to facilitate later incorporation of the Saccess data into the ground segment, should such a possibility be offered. 6.8 Supplementing the Swedish part of the collaborative ground segment with other types of data In addition to supplying the Swedish part of the Collaborative Ground Segment with Sentinel data, other types of data could be considered for inclusion. One data source that immediately springs to mind is Landsat 8 which has the same type of user licensing as the Sentinels. Landsat is already known to many, and has a similar target group to that envisaged for the CGSS. However, there are other types of data that could be useful to Swedish users from providers such as Airbus, Blackbridge and Digital Globe. Of course, including other types of data would not be without its own challenges. One drawback is that this implies management of different licensing models, some requiring extensive user account validation and classification of users. Nevertheless, it might be useful to explore the possibilities to include data from these and other data providers to complement the free Sentinel data. 14 SM-CGSS-FREP-10

15 SWEDEN DHuS ESA National mirror CGSS Collaborative Ground Segment Sweden Internet Landsat 8 ESA Long time archive x years of data Separate licens possible licens problem Airbus Blackbridge DigitalGlobe Figure 3 Potential CGSS links to other data sources Recommendation Spacemetric recommends that in addition to Sentinel data, the CGSS considers adding Landsat 8 data, as the two sensors are complementary and to a great extent share the same user community and have similar licensing. The benefits to Swedish users would be a single point of access for these two important missions. 6.9 Licensing Background In Spacemetric s brief, it is stated that the report should suggest solutions to any licensing issues that might arise in conjunction with usage of existing imagery in the Saccess database outside of the Nordic countries Discussion Some licensing aspects are covered above in the chapter on Saccess. Apart from these, there are, the team of investigators understand, some important points to be made regarding the licensing of Sentinel data for use in the Swedish Collaborative Ground Segment. These points are discussed below. Sentinel data is to have open and free access. Nevertheless, there are some conditions for the use of Sentinel data. The first and most important condition is perhaps unnecessary to point out, being that there must be an agreement in place between ESA and Sweden regulating Swedish use of Sentinel data. The Swedish National Space Board is the Swedish signatory body for the agreement with ESA and the draft agreement is understood to be under negotiation. According to indications from ESA 7, there is a requirement on Sweden to report on data usage. This appears not to necessitate the registering of Sentinel data users sourcing data from the Swedish part 7 Reported verbally by ESA and said to be defined in draft Member State agreements. 15 SM-CGSS-FREP-10

16 of the collaborative ground segment. Nor would there seem to be any need for validation of the users, as in the case of Saccess, since there is no requirement on Nordic domicile. Thus, the data handling is likely to be considerably less cumbersome, and thus considerably less expensive, than for Saccess. Having said this, it should also be noted that the information regarding ESA s demands for Sentinel data usage reports are not definitive. It would therefore seem prudent to go over these issues more thoroughly in conjunction with the negotiations with ESA. In this context, it seems apt to point out that services accessible without the need to register as a user often tend to attract by far more users than if user registration is needed Recommendations ESA s demands for Sentinel data usage reports should be clarified and analysed during the ongoing negotiations. The possibility to find solutions needing only minimal user data should be carefully explored Building blocks of the Swedish Collaborative Ground Segment Introduction If the requirement is just to secure access to Swedish users to the Sentinel data, a simple solution can be proposed, building on a straight-forward storage solution with access to files. The Swedish Collaborative Ground Segment, in such a case, would be no more than a one-to-one copy of the ESA Data Hub, a simple filestore with file download using FTP or http. But the consequence of such a system is that every user has to build their own infrastructure. The user base for such a solution would be very limited, and the value of the system in terms of benefit to society equally minimal. A more user-friendly approach is to offer easy access to services on top of the filestore. The raw files can be hidden (but with the possibility of access). The user can instead focus on their applications and on choosing the most relevant form of information selected from the hub s offering of mapprojected images, mosaics and composites. This level of user access enables users to either download data or, via services, to directly connect data to their applications, either from their desktops from mobile devices or even via a machine-to-machine interface CGSS Architecture The Swedish Collaborative Ground Segment is recommended to be built in two blocks (Figure 4). The foundation block should have many similarities with the simple file based system described above, with a simple file based approach. The second block building on this would be for the user services like search and quick look browse, map and image browse, ordering and subscriptions. 16 SM-CGSS-FREP-10

17 CGSS CGSS user services catalogue, order, subscription WMS, WCS, http, FTP Product 1 Product 2 CGSS long term archive Storage of system corrected images and metadata, reference data 200TB-800TB Figure 4 CGSS building blocks CGSS archive The CGSS archive would contain storage as well as functionality to harvest and ingest images and image metadata. It would be responsible for ensuring Sentinel data is collected and stored according to principles set up for the CGSS. The CGSS archive would have a fixed workflow (Figure 5). Images are harvested from the ESA Data Hub, ingested and then inserted into the catalogue and archived. This is characterised as a datadriven or push process. The process would in principle be initiated by ESA by making the image data available in the Data Hub, from where it is fetched by the CGSS. Each fetch operation will trigger a number of activities in the CGSS, for example actions specified in subscriptions. This will become the pulse of automated activities in the CGSS. Long term archive Catalogue Store metadata in a structured form for easy search ESA Data harvest Collect image data from ESA Data Hub Data ingest Collect metadata, validate and analyze for additional information Archive Store images in an structured form for easy access of pixels Figure 5 Long-term archive workflow Data harvest The basic functionality of the CGSS system is to mirror the contents of the ESA Data Hub according 17 SM-CGSS-FREP-10

18 to specific rules including area of interest and product type. This is a kind of harvest functionality where continuous requests are made to the ESA Data Hub using the ESA-supplied API. Each request is a query of the form are there any new image products in my area of interest?. The data harvest easily becomes a bottleneck, as it will be dependent on response performance in the ESA Data Hub at the same time that other Member State systems are making similar harvest requests and are competing for the same download bandwidth Data ingest The data ingest is where the metadata of the downloaded images are analysed. The metadata is interpreted and translated into information that can be added to the catalogue. The data ingest also contains processing steps with the aim to analyse and add information to the catalogue where the metadata supplied ESA is scanty or lacking. Examples of this are functionalities for improving cloud detection and geometric accuracy Cataloguing and archiving The catalogue makes it possible to search and browse images using the collated metadata. A typical query might be search for all images captured during the last 48 hours. The metadata contains specific attributes describing the images in terms of size, bands and format. It also contains geometric and radiometric attributes making it possible to present the image pixels in a geographic presentation on a map with accurate radiometric properties. The catalogue will range up to a few terabytes in size. The archive is where the image pixels are stored. The archive is recommended to be based on system-corrected images User services The user services make the Sentinel image data available to users in a user-friendly way. They are based on services (Figure 6). A service is easiest described as a user-driven access point. The user defines, in the form of a request, what is needed and the system replies with an answer. It is thus user-driven and the interaction is usually hidden in the user s application. In the CGSS, it is suggested to include four basic services: map service image service metadata service subscription service. The user can interact directly with the services via applications using machine-to-machine interfaces or via a web portal. The web portal should be an easy-to-use interface able to run without installation on the user side. Behind the scenes, the data served out to the users is computed using product components. For the CGSS, three basic product functionalities are suggested; for orthoimages, for mosaic and composite/ synthesis products and for time series. The Spacemetric team also suggests using a plug-in approach in order to make it possible for additional product functionalities to easily be deployed in the future. 18 SM-CGSS-FREP-10

19 User services Map service Ortho products Mosaic and synthesis products Time series products New products Image service Metadata service Subscription service User Production service Figure 6 User services architecture Map service This service takes a request from the user containing product (image, mosaic etc), area of interest, projection and output image format and turns it into a projected image. An example of industry standard for this service is OGC-WMS Image service This is basically the same as the map service but the returned image is in the raw system corrected geometry, i.e. sensor geometry. There is no industry standards for this kind of service but a modified variant of OGC-WMS can be used Metadata service The metadata service is a search and discover service to enable users to search for images and image products. The requests returns metadata Subscription service The subscription service is for automation. A subscription is basically a request to notify whenever there is a change. A change can be that there is newly ingested data covering a user specified area of interest. A subscription can for example be used to trigger product generation or product download Production service The production service is for delivery of larger datasets. When a user orders images or pieces of mosaics the production service helps in preparing the data and makes it available for download. The production service is a kind of file-based approach to the image and map services Ortho products The ortho product functionality is responsible for making orthoimages using input from archive and catalogue. The ortho generation is on-the-fly which means every request is computed on demand as a response to requests from the map service. Ancillary data such as DEM, geometric models and radiometric properties as well as coordinate system and file formats are applied on-the-fly in the 19 SM-CGSS-FREP-10

20 computation. Caching is recommended to avoiding recomputation of tiles. This is a heavy process and should enable scalability in terms of parallel processing Mosaic and synthesis products Multiple images are combined into mosaics and synthesis images. This product category is serviced out to end users using the map service. Virtualisation is recommended to enable fast publishing of mosaics. By virtualisation we mean that products should be defined in metadata and not as pre made products. This gives flexibility and makes it possible for end users to tailor the final product Time series products A time series is a stack of images or image mosaics where the focus is changes over time. This is a kind of product where the focus is coregistration of the images in the stack to make sure the pixels are on top of each other. This means the image to image registration needs to be on sub-pixel level. An image stack can be anything from a few images to hundreds of images New products This is a placeholder for future products. This is enabled using a plugin approach User registration Some form of user registration is an essential part of systems such as CGSS. In particular it facilitates a simplified user interaction (i.e. by remembering your preferences and previous sessions), it enables management of data licences and provides data for usage reporting. User registration makes it possible to store preferences, subscriptions and interaction histories. Tracking the frequency of tool usage or requests can help steer tailored user interfaces. Data licences often put restrictions on how imagery may be used. These restrictions are tightly connected to registered users. In this case the user accounts have to be validated. Usage reporting is essential for keeping track of system resources and how they have been employed. This is important if users are charged for using system resources. Reporting can also provide information on the usage patterns of individual users. Usage reporting is a more challenging when standards are used, such as OGC WMS, since there is usually no specification in the standards on how to do this. Using user registration for the purpose of managing licences makes the system more complex and will probably involve manual interaction from an operator. An example of this is the Saccess system where an operator has to validate every user account before it can be used in the system. In the long run, user registration can be expanded in functionality to incorporate handling of commercial image licences as a special case. The success of CGSS will be dependent on whether or not user registration is required. There are several examples where user services with a low user interest have been turned into success just by removing requirements on registration. A strong recommendation is to have user registration in the CGSS for the purpose of simplifying user interaction but not for licensing or statistics. The user registration would ideally be voluntary Usage reporting Usage reporting is the basis for statistics on system performance as well as how the system is used. The number of potential reporting parameters are enormous. To make the system manageable they must be kept to a reasonable number. 20 SM-CGSS-FREP-10