COPERNICUS SPACE COMPONENT DATA ACCESS 6TH DWH DATA SUPPLY USER SATISFACTION QUESTIONNAIRE - ACROSS COPERNICUS SERVICES PROVIDERS RESULTS

Transcription

1 COPERNICUS SPACE COMPONENT DATA ACCESS 6TH DWH DATA SUPPLY USER SATISFACTION QUESTIONNAIRE - ACROSS COPERNICUS SERVICES PROVIDERS RESULTS Prepared by Account Managers Reference COPE-GSEG-EOPG-RP Issue 1 Revision 0 Date of issue November 2014 Status Approved Document type Report Distribution level Public 1

2 Table of Contents 1 INTRODUCTION Quantitative indications about survey participation Document Overview Abbreviations Reference Documents QUESTIONNAIRE OVERVIEW Questionnaire Possible Answers Questionnaire Methodology ANALYSIS PER QUESTIONNAIRE ELEMENT FOR FP7 AND FOLLOW-UP PROJECTS CSC-DA Usage Frequency Data Access Portfolio Document (DAP) Data Ordering CDS-SCI Support Desk Data Access Service Performance Data Quality Web Portal Account Management Support Overall Perception of CSC-DA Services FEEDBACK FROM PUBLIC AUTHORITIES AND EU INSTITUTIONS FIFTH DWH RECOMMENDATIONS AND IMPLEMENTATION STATUS CONCLUSIONS ANNEX A

3 1 INTRODUCTION The Copernicus Space Component Data Access (CSC-DA) Questionnaire is one of the means within the ESA coordinated Copernicus Space Component Data Access (CSC-DA) project to collect, twice a year, feedback from Copernicus Service Providers (CSPs) as input to ensuring that the CSC-DA Service provides the best possible response to the Copernicus Service Providers needs and that improvements are implemented timely and consistently. An analysis of responses to the questionnaires is done per CSP and across CSPs. This report documents the results of the analysis of responses to the 6th DWH Data Supply User Satisfaction Questionnaire across CSPs covering the period December 2013 to June 2014; it includes recommendations for service improvements and reports on the implementation of recommendations made following the 5th DWH Survey. 1.1 Quantitative indications about survey participation Date of Survey: 03/07/2014 Survey period: December 2013 to June 2014 Number of FP7 and follow-up projects that have been contacted: 19 Number of FP7 and follow-up projects that provided feedback 1 : 12 Number of Public Authorities and EU Number of Public Authorities and EU Institutions that Institutions that have been contacted: 34 provided feedback: 5 Questionnaires have been sent out to all on-going FP7 and follow-up projects, Public Authorities and EU Institutions for which at least one member had signed the DWH Terms and Conditions by the date of the survey publication. The total number of FP7 and follow up projects contacted has decreased because several projects had terminated their activity before or at the very start of the observing period. 63% of the contacted FP7 and follow-up projects participated to the survey. Figure 1 shows the trend of survey participation over all performed surveys. Table 1 and table 2 show details distinguishing between FP7 and follow-up projects, Public Authorities and EU Institutions on survey participation. Participation of Public Authorities and EU Institutions has very slightly increased (by about 2%) but remains still low, as in the previous Survey, the same applies to participation from FP7 and follow-up projects which shows a small increase by 2%. 1 With at least one questionnaire 3

4 Figure 1 Trend Values - Percentage of Survey Participation of FP7 and follow-up projects FP7 and follow-up projects Survey participation CryoLand GIO EMS Ext Val LOBOS Prefer ERMES GIO-GLOBAL Lampre SAGRES G-NEXT GIOLand MACC II SenSyF G-SEXTANT ImagineS MYOcean2 Sensum GIO EMS InCreo NEREIDS Table 1. Overview of Copernicus Services Providers to whom the Questionnaire was sent and from whom a response was received. The highlighted FP7 and follow-up projects have provided feedback to the survey. Public Authorities and EU Institutions Survey participation EMSA HYDROMET TRAGSATEC University of Novi Sad EUSC Lantmäteriet DRAAF Julius Kühn- Insitut, Federal Institute for cultivated plants LUCAS University of Liege GMES-Masters Soil Conservation Service of Iceland ESA_VAE MF/DLR NW-FVA Specto Natura Ltd DG JRC National Meteorological Administration DALETT DG ENER Université catholique de JNCC Support Co Louvain Cemagref - UMR TETIS Delft University Institut national de Water and Land Resources Centre LEO 4

5 Northern Ireland Forest Service - Planning & Strategies Forestry Commission Technology FEGA State Institute for Nature Protection l'information géographique et forestière (IGN) Agency of Sustainable Development and Eurointegration - Ecoregions (ASDE) HVBG Table 2 Overview of Public Authorities and EU Institutions to whom the Questionnaire was sent and from whom a response was received. The highlighted CSPs have provided feedback to the survey. 1.2 Document Overview In the following sections the statistical results of the 6 th Data Warehouse (DWH) data supply user satisfaction questionnaire are presented, including Section 2 providing an overview of the questionnaire itself Section 3 presenting an analysis of the feedback from FP7 and follow-up projects per questionnaire element including the following information: o Assessment of user feedback and response on Poor to Satisfactory results o Recommendations o Across all questions (within the same questionnaire element) vertical (i.e. across all FP7 and follow-up projects) average value o Horizontal (i.e. across all responses from a same CSP) and vertical (i.e. across all FP7 and follow-up projects) aggregation per question within the same questionnaire element. Section 4 presents an analysis of the feedback from Public Authorities and EU Institutions Section 5 presenting the status of implementation of 4 th Survey recommendations. Section 6 presenting the overall conclusions 1.3 Abbreviations AM Account Manager AOI Area of Interest CDS Coordinated Data Access System CQC Coordinated Quality Control DAP Data Access Portfolio DWH Data Warehouse EOLI-SA Earth Observation Link Stand Alone ESA European Space Agency FTP File Transfer Protocol GCL GMES/Copernicus Client CCM(E) Copernicus Contributing Mission Entities GIO GMES/Copernicus Initial Operations GMES Global Monitoring for Environment and Security CSC-DA Copernicus Space Component Data Access CSP Copernicus Service Provider LLT Long Lead time MLT Medium Lead Time NRT Near Real-Time OTRS Open Ticket Request System 5

6 SCI SLT SPERF TPOC Copernicus Services Coordinated Interface Short Lead Time Service Providers Emergency Request Form Technical Point of Contact 1.4 Reference Documents [RD-1] Copernicus Space Component Data Access Portfolio: Data Warehouse DAP v2.8 [COPE-PMAN-EOPG-TN ] [RD-2] Fifth DWH Data Supply User Satisfaction Questionnaire: - Across GMES Services Providers Results [GMES-GSOP-EOPG-RP ] ] 6

7 2 QUESTIONNAIRE OVERVIEW The following subsections provide an overview of the questionnaire presented by questionnaire elements which most correspond to groups of CSC-DA Services, questionnaire possible answers and methodology. Questionnaire Elements CSC-DA Usage Frequency DAP Data Ordering CDS SCI Support Desk Data Access Service Performance Data Quality Web Portal Account Management Support Overall Perception of CSC- DA Services Table 3 Questionnaire Elements Questionnaire Questions 0a. How often do you access data 1a. How often do you consult the DAP Document 1b. How would you rate the usability of DAP Document 1c. Description of offered CSC-DA Services, i.e. access to data, ordering procedures, and helpdesk support 1d. Description of Data Offer for the identification of dataset of interest 1e. Accuracy of content for CORE datasets 1f. Accuracy of content for ADDITIONAL datasets 2a. How would you rate the usability of the Off-line Maps tool in your CSP Personal Area for submitting your data requests under standard ADDitional datasets 2b. How would you rate the usability of SPERF form for specifying your data requests under rush ADDITIONAL Datasets Emergency 2c. How would you rate the handling of data orders 3a. How would you rate the usability of the OTRS Ticketing System to place enquiries to the SCI Support Desk 3b. How would you rate the timeliness of enquiries handling 3c. How would you rate the clarity of information received 4a. How often do you use the GMES/Copernicus Client (GCL - EOLI- SA) tool for searching, browsing and downloading products 4b. FTP@CDS - access to standard datasets 4c. FTP@CDS - access to emergency basket 4d. FTP@GCM - access to emergency data on CCME FTP server 4e. FTP@GCM - access to systematic data deliveries for global or regional monitoring 4f. GMES/Copernicus Client (GCL - EOLI-SA) tool 5a. How would you rate the timeliness of data delivery within standard Additional Datasets 5b. How would you rate the timeliness of data delivery within Emergency Additional Datasets 5c. How would you rate the implementation of CORE Datasets (content, timeliness of data delivery) 6a. Quality of the interface with CDS-SCI or Account Manager 6b. Satisfaction about content of the answer 6c. Timeliness of the resolution 6d. How often do you consult the CQC Catalogue? 7a. How often do you visit the web site 7b. How would you rate the usability of the web site 7c. How would you rate your overall satisfaction with the web site 8a. Clarification of data offer 8b. Cooperation in resolving issues 8c. Support in selecting data or products 9a. How satisfied are you overall with the CSCDA Service you receive? 9b. How would you rate the quality of the products provided by CSCDA system? 9c. How would you rate the timeliness of the products? 9d. How satisfied are you with the way problems are solved within CSCDA? 7

8 2.1 Questionnaire Possible Answers All questions are multiple-choice. The possible answers to most questions were: N/A - Not applicable P - Poor S - Satisfactory G - Good E - Excellent For questions related to frequency possible answers were: N/A - Not Applicable D - Daily W - Weekly M - Monthly It is noted that analysis of questions may result as well in combinations of succeeding answers, i.e. resulting into P/S, S/G or G/E. 2.2 Questionnaire Methodology All eligible CSPs are invited to provide their feedback. Each CSP coordinator and TPOC (Technical point of contact) receives the User Satisfaction Questionnaire via and is required to fill in the form and send it back to the relevant Account Manager (AM) at gmes.account.manager@esa.int. Each recipient should fill in the form and is as well invited to extend it internally to colleagues to collect the most relevant feedback from each CSP. A date for the delivery of the form is normally indicated. The point of contact is the AM of the CSP who is responsible for the collection as well as for the analysis of the feedback. Results are then computed, analysed and commented by the AM and a presentation of the analysis including a trend comparison with previous surveys when applicable is prepared by AMs, presented during the CSPs Progress Meeting (when applicable) and published on the CSC-DA portal ( Since progress meetings with live-discussions on survey results are foreseen only for a limited number of CSPs, for most CSPs, a follow-up is organised through teleconferences in case provided answers result as Poor to Satisfactory. As support to these teleconferences a free text field (limited to 250 characters) allows CSPs to add information on their experience for each reply rated as Poor. 8

9 3 ANALYSIS PER QUESTIONNAIRE ELEMENT FOR FP7 AND FOLLOW-UP PROJECTS The following sections provide assessment of user feedback, response and recommendations. 3.1 CSC-DA Usage Frequency As in previous Surveys, results per project reflect normal expected behaviour for the last 6 months. MyOcean frequency is stable on Daily (NRT data access). G-NEXT and G- SEXTANT, after an increase in the last Survey, are stable on Weekly as well as the other CSPs. GIO-EMS External Validation has replied for the first time and declared usage frequency as Monthly. The purpose of this measure is to have an indication of the interaction of CSPs with CSC-DA Services which includes not only data access but also Web Portal consultation, reports and news, data ordering and SCI follow up support. The results are in line with the characteristics of the Projects, the stage of their lifetime and AM expectations. Table 4 CSC-DA Usage Frequency CSP CryoLand ERMES GIO-EMS GIO EMS Ext Validation GIO-Global G-NEXT G-SEXTANT ImagineS InCreo LOBOS MACC-II MYOcean2 CSC-DA Usage Frequency Few times a week Weekly Weekly Monthly Daily Weekly Weekly Few times a week Monthly Few times a week N/A Daily 3.2 Data Access Portfolio Document (DAP) Data Access Portfolio Document (DAP) How often do you consult the DAP Document? Across CSP average Good Across CSP average Monthly 9

10 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 2 DAP - Questionnaire Element by CSP Overall satisfaction about DAP is stable on Good Recommendations: Since a comment expressed during Progress Meeting by GIO-EMS has highlighted that DAP structure is perceived as too densely organized thus causing confusion for less experts users who have difficulty in finding the required information, it is foreseen that a thorough revision of the Document structure is performed for DAP release of the new DWH phase. 3.3 Data Ordering Data Ordering Across CSP average Satisfactory/Good 10

11 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 3 Data Ordering - Questionnaire Element by CSP The level of satisfaction decreased as a result of the use of the ordering interface by the Core Service Providers. The same comments were received from Cryoland, G-NEXT, G- SEXTANT, IMAGINES, GIO Global and LOBOS regarding the Off-line Maps interface: Slow response - The map to select an AOI is cartographically poor, does not display well not allowing to accurately define small areas (for VHR1 acquisitions). Coordinates pairs of the cursor on the map are displayed by Long/Lat instead of Lat/Long - Files with coordinates cannot be uploaded and displayed on the map - No changes to an order can be made if the submission is not accepted; also the user should be able to modify or cancel any order on line before it is sent to the CCME. - No auto-save function within order system (entered values disappear upon a refresh)+ - Application is slow and unintuitive for the user having to perform 2 actions for 1 scope: define a high level AOI on a cartographically poor map plus attach a file with the originally defined AOI. - Concerns about the fact that if the user inadvertently fails to attach the file, then the drawn high level AOI is taken into consideration and the tasking will be incorrect. - The sequence of required actions is also non intuitive, e.g. in order to check the AOI file attachment, it is first necessary to save the request. - The AOI file projection must necessarily be WGS84 whereas often user needs to use Mercator. Preference of UTM vs the required WGS84 was justified in G-NEXT progress meeting upon claim that the overall size of AOI and therefore the km2 charged to quota (in particular for optical VHR data) is bigger with WGS84. - The notifications of product availability are only sent to the requester having logged in. Need to include other addresses in copy of the notifications. - The automatic notifications, at time, are received hours after the image has already been downloaded from the ODA. - Offline Mapping facility being unavailable on one occasion. 11

12 - Absence of regular updates concerning the success/failure of a planned acquisition and, if a failure, when the next acquisition would occur. ESA: Off line Maps interface: small changes have been implemented during the reporting period, while as announced in the last questionnaire report, significant improvements of both interface and selected map server will be incorporated in CDS V3 currently under implementation. Currently, upon the rare occurrence of interface unavailability, the back-up option exists to send the order via with inclusion of AOI shapefile. Projection of Shapefile: Claimed differences in AOIs size between UTM vs the required WGS84 will be investigated. Product availability notification: product notifyer is triggered currently every 8 hours, but the latency can be reduced. The possibility of adding more addressees than the requester will be investigated. On-line emergency ordering form: comments gathered during Progress meetings with GIO-EMS and G-NEXT/G-SEXTANT indicated the need for: A on-line repository holding a sort of summary/dossier recapping the most relevant information per activation (i.e. SPERFs, SRTs). More information in the SRT on products delivery schedule. The possibility among the project to share submitted SPERFs. The first point will be addressed in CDS V3; for the second point, the information is passed on if known from CCME. 3.4 CDS-SCI Support Desk CDS SCI Support Desk Across CSP average Good 12

13 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 4 CDS-SCI Support Desk - Questionnaire Element by CSP The level of satisfaction reflects mainly the limitations of the ticketing system. The following comments were received from IMAGINES, GIO-Global-Land, Increo, LOBOS and Cryoland: - OTRS Ticketing system should allow addition of recipients other than SCI and show the history. - Difficult to follow conversation threads, time consuming and clumsy setup, temptation to reply directly to ticket responses instead of logging in to respond. - The absence of direct contact/reply via mail with SCI is a cause of regret - Delayed answers regarding a login issue but very reactive for providing historic monthly status reports. ESA: Ticketing system: the sharing of tickets has been implemented during the reporting period and the current ticketing system will be phased out with CDS V3. A delayed answer on the login issue was caused by difficulties in finding the root cause of the problem. 3.5 Data Access Data Access Across CSP average Satisfactory/Good 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory 13

14 Figure 5 Data Access - Questionnaire Element by CSP The following comments were received from G-NEXT, LOBOS and ERMES: - FTP@GCM - access to systematic data deliveries for global monitoring. The way in which Proba-V are provided could be improved for what concerns the "clarity" of the folders structure. Understanding what data are provided in the FTP folders proved to be quite difficult. - Copernicus Client (GCL - EOLI-SA) tool: functionality to retrieve imagery from shop carts received with delivery notifications are very unreliable. Download functionality has also been unreliable and speed highly variable. ODA/FTP is far more intuitive. - The client software shows several problems (e.g. unavailability of quickviews for "archived" products, impossibility to filter out not-downloadable "Acquired" scenes/products) - The threshold on the number of searchable collections prevents user from obtaining an effective overview of data availability. The addition of a new searches in GCL/ EOLi-SA tool for All SAR VHR and All SAR HR collections, and the same for optical datasets is desirable. ESA: GCL Client: The GCL/EOLi-SA tool will be replaced by a new catalogue interface in CDS V3 that will implement flexible searches, products download via their URLs, subscription to a catalogue search and automatic download, possibility to request hosted processing, etc. In the meantime, upon last questionnaire feedback, new search options have been defined and will be implemented within November. A beta version of new simplified collections searches was shown during the Progress Meeting with G-NEXT/G-SEXTANT at the end of October. The advanced search in CGL/EOLISA is also under analysis for the filtering of acquired items. Functionality of GCL to retrieve images from Shopcarts sent via product readiness notifications: user specific issue was investigated and solved at the end of May Organisation and structure of Proba-V server at VITO: user comment is taken on board and a more meaningful naming of directories and sub-directories is being redesigned for implementation by end of Service Performance Service Performance Across CSP average Good 14

15 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 6 Service Performance - Questionnaire Element by CSP For GIO-EMS in particular, data delivery timeliness does not yet meet users expectations. Users concerns are why products once acquired are not delivered more rapidly. ESA: as mentioned in the last survey results, improvements were already implemented where possible, limitations remain but are mainly dependent on the space segment capabilities. Technical constraints will be again discussed with CCMEs for the next phase. Constraints remaining in the new phase of the DWH will be documented in the DAP. It is worth to note that as a result of the flexibility and scenarios used during emergency activations, the statistics about average timeliness have to be carefully analysed (e.g. delay till sensing due to weather forecast). 3.7 Data Quality Data Quality How often do you consult the CQC Catalogue Across CSP average Good/Excellent Monthly 15

16 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 7 Data Quality - Questionnaire Element by CSP The satisfaction has improved. No issues registered for this section. 3.8 Web Portal Web Portal How often do you visit the web site Across CSP average Satisfactory/Good Monthly 16

17 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 8 Web Portal - Questionnaire Element by CSP During the Progress Meeting G-SEXTANT commented that information is difficult to find on the CSCDA (confusing structure and unclear naming of sections). ESA: A major review of the web site in foreseen in CDS V3 and Service Provider has been requested to provide detailed comments. 3.9 Account Management Support Account Management Support Across CSP average Good/Excellent 17

18 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 9 Account Management Support - Questionnaire Element by CSP Ratings are stable on Good/Excellent Overall Perception of CSC-DA Services Overall Perception of CSC-DA Services Across CSP average Good 18

19 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 10 Overall Perception of CSC-DA Services - Questionnaire Element by CSP The overall perception is stable on Good since the 2nd survey. The POOR rating associated to the overall perception of how CSDA project solves problems is not specifically explained during GIO-EMS Progress Meeting therefore the spontaneous conclusion is the need for more regular contacts and status reviews between AM and JRC in order to tackle punctually and in detail possible issues. 19

20 4 FEEDBACK FROM PUBLIC AUTHORITIES AND EU INSTITUTIONS As illustrated in Table 2, the responsiveness of Public Authorities and EU institutions has slightly increased from the last Survey, now 14% vs 13% of last Survey. This participation level is still low considering that the number of subscribed datasets has increased by 132% and the number of users has increased by 48%. 20

21 5 FIFTH DWH RECOMMENDATIONS AND IMPLEMENTATION STATUS The analysis of the Fifth DWH Survey had produced a list of recommendations for improvements focused on the following topics: Data Ordering Changes to the Off line Mapping tool : Product type, processing level should be selectable options Balloons popping out during request submission containing either URLs to user manuals/guidelines or tips on how to do it Calendar option for entering dates Multiple acquisition windows input ruled by a form and not in the notes KML/other shapefiles upload and visualisation on the Map Dataset description offered next to the dataset code in the list of selectable datasets Minor changes have been implemented to the interface while most of the above requirements were taken up for CDS v3. Data Access GCL/EOLi-SA: the possibility to increase the number of searchable collections should be investigated and possibly new search options be added in the same fashion as the existing one: ALL datasets (All SAR VHR, All SAR HR, All Optical VHR and All Optical HR) This improvement was taken on board and it is currently being implemented in Nov FTP@GCM In the Emergency scenario, the association of delivery s of some CCMs (i.e. DigitalGlobe) to the specific sub-id is to be improved. The s will be improved once EUSI is integrated to the CDS (before the end of 2014) 21

22 6 CONCLUSIONS As visualised in Figure 11, the overall inter groups analysis shows that there are 3 Services that are rated as Satisfactory/Good: Web Portal, Data Ordering and Data Access. 4 Services are rated as Good: Overall perception, Service Performance, SCI Support and DAP whereas 2 Services are rated as Good/Excellent: Account Management support and Data Quality. 5: Good/Excellent, 4: Good, 3: Satisfactory/Good, 2: Satisfactory, 1: Poor/ Satisfactory Figure 11 Questionnaire item groups across all CSPs The trend analysis from 1 st Survey to 6 th Survey (Table 5) shows that users have recognised the improvement for Data Quality to Good/Excellent after having been stable on Good for the first 5 surveys. Data Ordering and Data Access has slightly decreased to Satisfactory/Good as well as SCI support to Good. The remaining services are confirmed on Good and Good/Excellent as per last Survey. It is to be noted that Web Portal appreciation has not changed since the beginning and is stable on Satisfactory/Good. Table 5 Trend Analysis From 1st to 6th Survey G/E: Good/Excellent, G: Good, S/G: Satisfactory/Good, S: Satisfactory, P/S : Poor/ Satisfactory Survey #1 - Across CSP average Survey #2 - Across CSP average Survey #3 - Across CSP average Survey #4 - Across CSP average Survey #5 - Across CSP average Survey #6 - Across CSP average DAP S/G S/G G G G Good Data Ordering S/G S/G S/G G G Satisfactory/Good CDS SCI Support G G G G G/E Good Desk Data Access S/G S/G G G G Satisfactory/Good Service Performance S/G S/G S/G G G Good Data Quality G G G G G Good/Excellent Web Portal S/G S/G S/G S/G S/G Satisfactory/Good Account Management G G/E G/E G/E G/E Good/Excellent Support Overall Perception of CSC-DA Services S/G G G G G Good 22

23 Based on the questionnaire results, teleconferences and progress meetings organised with the CSPs, the main topics where efforts should be concentrated are: DAP document The document should be improved and more importantly simplified so that less expert users can easily locate the required information. A review of the DAP is on-going for the DWH Phase 2 release. Data Ordering Major improvements are in the design and implementation of CDS V3. Data Access GCL/EOLi-SA: additions and changes to the searchable collections are being implemented in Nov The tool will be replaced with ngeo in CDS V3. CDS SCI Support Desk OTRS: the tool will be phased out in CDS V3 and the interface to users will be based on s Web Portal The Web Portal will be reviewed in the frame of CDS V3 implementation A first version of CDS V3 is planned to be transferred in operations in March 2015; progressive release of functionalities is foreseen over one year. 23

24 7 ANNEX A This section presents the CSP rating records per questionnaire element, i.e.: Across all questions (within the same questionnaire element) vertical (i.e. across all FP7 and follow-up projects) average value Horizontal (i.e. across all responses from a same CSP) and vertical (i.e. across all FP7 and follow-up projects) aggregation per question within the same questionnaire element. Table 6 Rating Abbreviations G/E: Good/Excellent G: Good S/G: Satisfactory/Good S: Satisfactory P/S: Poor/ Satisfactory M: Monthly W: Weekly F: Few days a week D: Daily Table 7 6th Survey - CSP Ratings Across CSP average CryoLand ERMES G-NEXT G-SEXTANT GIO-Global GIO EMS GIO EMS Ext Validation ImagineS InCreo LOBOS MACC-II MYOcean2 1. a. How often do you consult the DAP Document? 1.b. How would you rate the usability of DAP Document 1.c. Description of offered CSC-DA Services, i.e. access to data, ordering procedures, and helpdesk support 1.d. Description of Data Offer for the identification of dataset of interest 1.e. Accuracy of content for CORE datasets 1.f. Accuracy of content for ADDITIONAL datasets M M W M M M W M M M M N/A M S/G G/E S G S/G G G S G S S S G G G/E G G/E G G G G G G S G G G G/E G G/E G/E G G G G S S G G S/G N/A S N/A N/A G N/A G G S N/A G G G G/E N/A G/E G/E G G G G G S N/A G DAP: Vertical Aggregation on CSP Level G G/E S/G G/E G G G G G S/G S S/G G 2a. How would you rate the usability of the Off-line Maps tool in your CSP Personal Area for submitting your data requests under standard ADDitional datasets 2b. How would you rate the usability of SPERF form for specifying your data requests under rush ADDITIONAL Datasets Emergency 2c. How would you rate the handling of data orders S P/S N/A S S P/S N/A S G N/A N/A G N/A N/A G N/A P/ S N /A S S N/A G N/A N/A N/A N/A G G N/A G/E S/G G G G G S G N/A G/E 24

25 Across CSP average CryoLand ERMES G-NEXT G-SEXTANT GIO-Global GIO EMS GIO EMS Ext Validation ImagineS InCreo LOBOS MACC-II MYOcean2 Data Ordering : Vertical Aggregation on CSP Level 3a. How would you rate the usability of the OTRS Ticketing System to place enquiries to the SCI Support Desk 3b. How would you rate the timeliness of enquiries handling 3c. How would you rate the clarity of information received CDS SCI Support Desk : Vertical Aggregation on CSP Level 4a. How often do you use the GMES/Copernicus Client (GCL - EOLI-SA) tool for searching, browsing and downloading products 4b. FTP@CDS - access to standard datasets 4c. FTP@CDS - access to emergency basket 4d. FTP@GCM - access to emergency data on CCME FTP server 4e. FTP@GCM - access to systematic data deliveries for global or regional monitoring 4f. GMES/Copernicus Client (GCL - EOLI- SA) tool Data Access: Vertical Aggregation on CSP Level 5a. How would you rate the timeliness of data delivery within standard Additional Datasets 5b. How would you rate the timeliness of data delivery within Emergency Additional Datasets 5c. How would you rate the implementation of CORE Datasets (content, timeliness of data delivery) Service Performance : Vertical Aggregation on CSP Level 6a. Quality of the interface with CDS-SCI or Account Manager 6b. Satisfaction about content of the answer S/G S N/A G S S G S/G S S S/G N/A G S/G P/S G G G P/S N/A G G S G G/E G/E P/S G G/E P/ S P/ S P/S P/S N/A G/E S G N/A G/E G/E G/E G G/E G/E G G G/E G G G N/A G/E G S/G G G/E G/E P/S G G/E M M M M N/A N/A N/A M P/ S N /A S S/G N/A G/E M M N/A N/A G G/E S G G G N/A S G G G N/A G/E G N/A N/A G N/A N/A N/A N/A G N/A N/A G N/A N/A G N/A G G/E P/S N/A N/A G/E N/A N/A N /A N /A N /A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A G S S P/S S G S N/A S S S P/S N/A N/A S/G G/E P/S S/G G G G S S/ G S/G S N/A G/E G S N/A G/E G G N/A G G G G N/A G G/E N/A N/A G/E N/A N/A S N/A G N/A S N/A N/A G/E N/A G N /A N /A N/A N/A N/A N/A G N/A N/A G G S S G/E G G/E S G G G G N/A G G/E G N/A G/E G G/E E G G/ E G G/E N/A G/E G/E G/E N/A G/E G/E G G G G G G/E N/A G/E 6c. Timeliness of the resolution G/E G N/A G/E G/E G E G G G G N/A G/E 6d. How often do you consult the CQC Catalogue? Data Quality: Vertical Aggregation on CSP Level M M N/A N/A N/A N/A N/A N/A G/E G/E N/A G/E G/E G/E E G N /A G /E M N/A N/A N/A G G/E N/A G/E 7a. How often do you visit the web site M M W W W M M M M M F N/A M 7b. How would you rate the usability of the web site S/G S S G S S S G S S G S G 25

26 Across CSP average CryoLand ERMES G-NEXT G-SEXTANT GIO-Global GIO EMS GIO EMS Ext Validation ImagineS InCreo LOBOS MACC-II MYOcean2 7c. How would you rate your overall satisfaction with the web site Web Portal : Vertical Aggregation on CSP Level S/G S S G S/G S S G S S S S G S/G S S G S S S G S S S/G S G 8a. Clarification of data offer G/E G/E G G/E G/E G/E G G 8b. Cooperation in resolving issues G/E G/E G/E G/E G/E G/E G G 8c. Support in selecting data or products G/E G/E G G/E G/E G/E G G Account Management Support : Vertical Aggregation on CSP Level 9a. How satisfied are you overall with the CSCDA Service you receive? 9b. How would you rate the quality of the products provided by CSCDA system? 9c. How would you rate the timeliness of the products? 9d. How satisfied are you with the way problems are solved within CSCDA? Overall Perception of CSC-DA Services: Vertical Aggregation on CSP Level G/E G/E G/E G/E G/E G/E G G G/ E G/ E G/ E G /E G/E G/E G G/E G/E G/E N/A G/E G/E G/E N/A G/E G/E G/E G G/E G G S G/E G G G G G S G S G/E G G/E S G G G G G G G G N/A G/E G G N/A G/E G G S G G G G N/A G G G G G/E G S E G S G G N/A G/E G G S/G G/E G S/G G G S/ G S/G G S G/E 26