D3.1 Guidelines to the assessment the sustainability and maturity of building blocks

Transcription

1 Submitted to the EC on 01/10/2013 COMPETITIVENESS AND INNOVATION FRAMEWORK PROGRAMME ICT Policy Support Programme (ICT PSP) Project acronym: e-sens Project full title: Electronic Simple European Networked Services ICT PSP call identifier: CIP-ICT-PSP ICT PSP main theme identifier: CIP-ICT-PSP Basic Cross Sector Services Grant agreement n : D3.1 Guidelines to the assessment the sustainability and maturity of building blocks Deliverable Id : D3.1 Deliverable Name : Guidelines to the assessment the sustainability and maturity of building blocks Version : v 1.0 Status : Final Dissemination Level : Public Due date of deliverable : M6 Actual submission date : Work Package : WP3, Task 3.1 Organisation name of lead partner for this deliverable : Ministry of Economic Affairs and Communications of Estonia Author(s): Jaak Tepandi Partner(s) contributing : DIGST.DK Denmark; JM NRW Germany; EISA Estonia - Ministry of Economic Affairs and Communications; SGMAP France - MINISTERE DE LA JUSTICE; Tudor Luxembourg; NL-MEA Netherlands - (TNO), Forum Standaardisatie; Difi Norway; TUBITAK Turkey Abstract. Deliverable D3.1 is intended to assess various types of building blocks. It presents a procedure for assessment, criteria and information to be used in the four assessment steps, and organisational aspects of assessment. The assessment procedure comprises proposal, consideration, assessment and recommendation D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 1

2 steps. A research update on already existing studies is provided in an Annex. The deliverable is based on the CAMSS methodology, ADMS, the e-sens WP6 deliverable D6.1, suggestions from e-sens stakeholders, standards, and other sources. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 2

3 History Version Date Changes made Modified by D3.1 Guideline principles Jaak Tepandi Guideline principles are extended and updated to a draft Jaak Tepandi Guideline; comments of WP3 participants have been incorporated The draft Guideline criteria have been elaborated, Jaak Tepandi extended, and updated according to suggestions from the WP3 Brussels meeting on and teleconference on New presentation of the assessment procedure has been Jaak Tepandi elaborated. Assessment criteria from different sources have been merged. New template for the report has been applied. The draft Guideline has been extended and updated throughout the text according to suggestions from T3.2 and other WP3 participants The proposal information/criteria have been aligned with Jaak Tepandi WP6. The Guidelines have been closer aligned with the e- SENS deliverable template. Proposals and comments of WP3, WP5, WP6, and other e-sens project participants have been taken into account Conformance to e-sens template has been improved. Jaak Tepandi Chapter for research update and input for the analysis has been added. Flexibility of the Guidelines has been emphasized. Proposals and comments from WP3 and other e-sens project participants have been inserted. The Guidelines have been submitted for the e-sens Deliverable Revision Cycle Proposals and comments for the previous version from Jaak Tepandi WP1, WP3, WP5, and WP6 participants have been incorporated Final check by WP1 Carsten Schmidt D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 3

4 Table of Contents HISTORY... 3 TABLE OF CONTENTS... 4 LIST OF FIGURES... 7 LIST OF TABLES... 8 LIST OF ABBREVIATIONS AND GLOSSARY... 9 EXECUTIVE SUMMARY INTRODUCTION SCOPE AND OBJECTIVE OF THE GUIDELINES WP3 GENERAL OBJECTIVES AND VISION METHODOLOGY OF WORK CAMSS ADMS Other sources RELATIONS TO INTERNAL E-SENS ENVIRONMENT Relations to e-sens Work Packages Reviewing and commenting on the Guidelines RELATIONS TO EXTERNAL E-SENS ENVIRONMENT QUALITY MANAGEMENT RISK MANAGEMENT LEGAL ISSUES STRUCTURE OF THE DOCUMENT THE ASSESSMENT PROCEDURE AND TARGETS THE ASSESSMENT PROCEDURE THE TARGETS OF ASSESSMENT PROPOSAL AND CONSIDERATION STEPS: DOCUMENTATION OF FORMAT THE PROPOSAL STEP Complete list of proposal information/criteria The highly important proposal information/criteria THE CONSIDERATION STEP THE ASSESSMENT STEP D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 4

5 4.1 INTEGRATION AND USE OF ASSESSMENT CRITERIA Integration of Assessment Criteria from Various Sources Use of Assessment Criteria STANDARDISATION CRITERIA Maturity Openness Intellectual property rights Life cycle, maintenance, service levels, security CRITERIA FOR ALIGNMENT WITH EXISTING POLICY FRAMEWORKS Basic alignment criteria Applicability Potential BUSINESS NEED CRITERIA Basic business need criteria Market support THE RECOMMENDATION STEP: CRITERIA AND CLASSIFICATION RECOMMENDATION CRITERIA PROPOSED CLASSIFICATION ORGANIZATIONAL ASPECTS OF ASSESSMENT A QUESTIONNAIRE FORM FOR THE ASSESSMENT OF EUROPEAN BUILDING BLOCKS DEALING WITH CURRENT LSPS THAT DO NOT COMPLY TO THE CRITERIA UPFRONT CONCLUSION REFERENCES CONTRIBUTORS APPENDIX 1. RESEARCH UPDATE AND INPUT FOR THE ANALYSIS INTERIM EVALUATION OF THE ISA PROGRAMME THE FEASIBILITY AND SCENARIOS FOR THE LONG-TERM SUSTAINABILITY OF THE LSPS STUDY ON ANALYSIS OF THE NEEDS FOR CROSS-BORDER SERVICES SPOCS D3.1 ASSESSMENT OF EXISTING E-DELIVERY SYSTEMS & SPECIFICATIONS SPOCS D4.1 ASSESSMENT OF SELECTED NATIONAL APPROACHES AND POSSIBLE SOLUTIONS STORK D5.1 EVALUATION AND ASSESSMENT OF EXISTING REFERENCE MODELS AND COMMON SPECS STORK D6.6 EVALUATION REPORT APPENDIX 2. PROPOSAL INFORMATION/CRITERIA BASED ON CAMMS APPENDIX 3. PROPOSAL INFORMATION/CRITERIA BASED ON WP6 DELIVERABLE D D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 5

6 13 APPENDIX 4. THE CAMSS ASSESSMENT CRITERIA D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 6

7 List of Figures Figure 1. The procedure for assessment with main stakeholders (Executive Summary) Figure 2. The procedure for assessment with main stakeholders D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 7

8 List of Tables Table 1. Abbreviations and glossary Table 2. Development of the Guidelines and collaboration with other WPs Table 3. Quality Management for the Guidelines Table 4. Risk Management for the Guidelines Table 5. The complete list of proposal information/criteria based on CAMMS, D6.1, and ADMS Table 6. The highly important proposal information/criteria in the recommended order of presentation Table 7. Consideration criteria Table 8. Assessment criteria for standardization: maturity Table 9. Assessment criteria for standardization: openness Table 10. Assessment criteria for standardization: intellectual property rights Table 11. Assessment criteria for standardization: life cycle Table 12. Assessment criteria for alignment: basic Table 13. Assessment criteria for alignment: applicability Table 14. Assessment criteria for alignment: potential Table 15. Assessment criteria for business need: basic Table 16. Assessment criteria for business need: market support Table 17. Knock-out criteria Table 18. Recommendation criteria Table 19. Dealing with current LSPs that do not comply to the criteria upfront Table 20. Contributors to D Table 21. Appendix. Proposal information/criteria based on CAMMS Table 22. Appendix. The CAMMS assessment criteria D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 8

9 List of Abbreviations and Glossary Acronym A2A ABB ADMS ADMS.SW BB BCSS BOMOS CAMSS CIP e-codex e-sens e-id epsos EIA EIF (F)RAND Guidelines H/R Explanation Administration to Administration Architecture Building Block Asset Description Metadata Schema Asset Description Metadata Schema for Software Building Block, represents a (potentially re-usable) component of business, IT, or architectural capability that can be combined with other building blocks to deliver architectures and solutions (a TOGAF 9 definition) Basic Cross Sector Services Management and Development Model for Open Standards Common Assessment Method For Standards And Formal Specifications. Final Draft Revision of CAMSS. Version 1.0, March 2012 Competitiveness and Innovation Programme Online access to judicial procedures for claimants, defendants and legal professionals (e-justice Communication via Online Data Exchange) Electronic Simple European Networked Services Electronic identity Smart Open Services for European Patients (Cross-border access to patient information online) European Interoperability Architecture European Interoperability Framework (Fair,) reasonable, and non-discriminatory terms Guidelines to the assessment the sustainability and maturity of building blocks (the current document) Highly important / recommended D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 9

10 HBB High Level Building Block IDABC Interoperable Delivery of European egovernment Services to public Administrations, Businesses and Citizens ISA ISMS LSP OpenBRR PEPPOL SBB SLA SPOCS STORK Target assessment of Interoperability Solutions for European Public Administrations programme Information security management system Large Scale Pilot Open Software Business Readiness Rating Interoperable eprocurement solutions, such as e-invoices and e-signatures for important documents Solution Building Block Service Level Agreement Simple Procedures Online for Cross- Border Services (Online Points of Single Contact to help businesses expand into other countries) Secure identity across borders linked (Electronic identity for easier access to public services) An artifact (typically in a TOGAF 9 sense: "Artifact - an architectural work product that describes an aspect of the architecture"), submitted for assessment in the proposal step, to be assessed through the consideration, assessment, and recommendation steps TA, Annex TOA WP Technical e-sens Technical Annex v3.0 See "Target of Assessment" (used in tables) Work Package Table 1. Abbreviations and glossary D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 10

11 Executive Summary The aim of the e-sens (Electronic Simple European Networked Services) project is to provide generic interoperable solutions for cross-border public services in Europe. e-sens is another Large Scale Pilot project launched by the European Commission to support the realisation of European interoperability policies. All LSPs already launched facilitate the use of innovative technologies for deployment of EU-wide services in selected areas and, in turn, the development of a digital single market. The existing and already completed Large Scale Pilots have already proven that providing cross-border services can be made simpler. In numerous domains, technical building blocks have been developed and piloted, which enable seamless cross-border services respecting all the various challenges and requirements faced. The essence of the new e-sens pilot is to consolidate and solidify the work achieved to date, to industrialise the solutions, and to extend their potential to more and different domains. The goal of e-sens Work Package 3 is to pave the way for sustainability and long-term governance of the LSP building blocks and their usage and interoperability within all European Member States and Associated Countries. WP3 aims to present proposals for sustainable building blocks such as e-id, e-signatures, e-documents, e-delivery, that have emerged from the Large Scale Pilots relevant to the e-sens project. This proposal should support competitiveness, openness for future technologies, and interoperability. Task T3.1 of the Work Package 3 supports presentation and assessment of building blocks. Its main deliverable is Guidelines to the assessment the sustainability and maturity of building blocks (referred to as "Guidelines", which is the current document). The objective of the Guidelines is to propose a documentation of format and defining criteria for the maturity and sustainability assessment of building blocks in close cooperation with WP3 (in particular Task 3.2 Sustainability assessment ), WP 5, and WP 6. Task 3.1 will be active from M1 to M6. The current version of the Guidelines builds upon existing work carried out in the European Interoperability Framework and is based on the e-sens Technical Annex v 3.0, the CAMSS methodology (Final Draft Revision of CAMSS, Version 1.0, March 2012), the ADMS (Asset Description Metadata Schema, Specification Version 1.00), discussions with e-sens project participants, and various other e-sens project materials. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 11

12 Following the actual e-sens Technical Annex and the CAMSS methodology, the current document uses the assessment procedure that comprises proposal, consideration, assessment, and recommendation steps as follows. 1. In the proposal step, a target of assessment (for example, an Architecture Building Block or High level Building Block, including support artifacts such as guidelines) is provided to Task 3.2 by WP6 Architectural Board. The target of assessment is provided using the proposal information/criteria. The proposal information/criteria provide general information about the proposed target of assessment, its status, items provided for assessment, and other. 2. In the consideration step, consideration criteria are used before the actual assessment, to validate information received and relevance of the proposal. 3. In the assessment step, the criteria used are categorised under standardisation, alignment with existing policy frameworks, and business need. Additional information will be sought from other Work Packages and external stakeholders. 4. In the recommendation step, recommendation criteria are applied to conclude with a classification (Discarded, Observed, Accepted, Recommended and Mandatory) of the target of assessment. Depending on the e-sens mandate, it may be agreed to avoid the "Mandatory" recommendation in the assessments. This classification will be reported back to WP6 Architectural Board for using by other Work Packages. Figure 1 captures the primary process flow. The Guidelines, including the sets of criteria provided, are intended for assessing various types of building blocks. During a specific assessment, relevant criteria are selected depending on the target submitted for evaluation - the Guidelines are flexible and should help with assessment. Assessments may be done iteratively, in cooperation with WP6, WP5, and other stakeholders. The document starts with the background given in the introduction. Section 2 presents a procedure for assessment and an overview of the targets to be assessed. Section 3 describes the criteria for the proposal and consideration steps. Section 4 is devoted to the assessment step and criteria, section 5 - to the recommendation step. Section 6 presents the organizational aspects of assessment. To help conducting future assessments, results of already existing assessments are identified and analysed for re-use in Appendix 1. The Guidelines is a living document that can be updated after changes in underlying materials, due to experience gained in pilot assessments, as a result of new agreements, and other. It is proposed that a review of this document takes place 3 months before each update of D3.2" Assessment on the maturity of building blocks" - i.e., in months M15, M27, and M33. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 12

13 Figure 1. The procedure for assessment with main stakeholders (Executive Summary) D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 13

14 1 Introduction 1.1 Scope and objective of the Guidelines Task T3.1 of the Work Package 3 supports presentation and assessment of building blocks in line with the European Interoperability Architecture 1 (EIA) and the European Interoperability Framework 2 (EIF). This work is done in close cooperation of WP3 with WP5, and WP6. Concretely, this subtask develops guidelines and a procedure to assess the status of existing as well as new core building blocks, such as e-id, e-signatures, e-documents, and e-delivery. The main deliverable of Task T3.1 is Guidelines to the assessment the sustainability and maturity of building blocks ("Guidelines", the current document). Task 3.1 will make it easier to focus on the generic, modular and re-usable character of these building blocks. The objective of the Guidelines is to propose a documentation of format and defining criteria for the maturity and sustainability assessment of building blocks in close cooperation with WP3 (in particular Task 3.2 Sustainability assessment ), WP 5, and WP 6. Task 3.1 is active from M1 to M6. The Guidelines are intended for assessing various types of building blocks. A procedure and a questionnaire form are presented for the assessment of building blocks. Deliverable 3.1 is a living document that has to be updated after changes in underlying materials, due to experience gained in pilot assessments, as a result of new agreements, and other. 1.2 WP3 General Objectives and Vision The goal of e-sens Work Package 3 is to pave the way for sustainability and long-term governance of the LSP building blocks and their usage and interoperability within all European Member States and Associated Countries. WP3 aims to present proposals to sustain core building blocks such as e-id, e-signatures, e- Documents, e-delivery, that have emerged from the Large Scale Pilots relevant to the e-sens 1 European Interoperability Architecture, 2 European Interoperability Framework (EIF) for European public services, D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 14

15 project. These proposals should support competitiveness, openness for future technologies, and interoperability. 1.3 Methodology of Work The methodology for developing the Guidelines builds upon existing work carried out in the European Interoperability Framework 3 and is based on the CAMSS methodology 4, the ADMS 5, discussions with e-sens project participants, and various other e-sens project materials. CAMSS has been proposed as a reference process and criteria for Member States to perform assessments of formal specifications and therefore serves as a good resource for the Guidelines. It is also identified as a starting point for the Guidelines in the Technical Annex. The Guidelines extend and modify the CAMMS methodology in the scope, detail and process, for example introducing categories of criteria required in the Technical Annex but missing in CAMSS. During the assessment (starting from the proposal step) it is necessary to describe the interoperability assets. This is the main task of the Asset Description Metadata Schema. For this reason, ADMS was used as another source for the Guidelines. Development of the Guidelines comprised the following the process and collaboration activities with other WPs. Deadline / duration Activities Deliverables Start of the task T Development of main principles for the Main principles for the Guidelines. Preliminary discussions with Guidelines participants Discussion of the main principles with Adjustment of main participants (in Brussels) principles May-June 2013 Preparation and discussion of the preliminary draft of the Guidelines via and / or teleconference Preliminary draft of the Guidelines 3 European Interoperability Framework (EIF) for European public services, 4 Common Assessment Method For Standards And Formal Specifications (CAMSS). Final Draft Revision of CAMSS; Version 1.0, March ADMS: Asset Description Metadata Schema; Specification Version Release date 18/04/2012 D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 15

16 Deadline / duration Activities Deliverables Discussion of the preliminary draft of the Adjustment of the Guidelines (in Tallinn, July 2013) preliminary draft Preparation and discussion of the draft version Draft version of the of the Guidelines via and / or Guidelines teleconference (be-weekly teleconferences of WP3 and Task 3.2, teleconferences of the WP3/WP5/WP6 joint Summer Task Force, private communications etc.) Preparation of the intermediate draft and final versions and their discussion as before Draft and final versions of the Guidelines Table 2. Development of the Guidelines and collaboration with other WPs The assessment methodology is given by the assessment procedure, criteria, and guidelines in the main body of this document. It follows the level of detail for assessment methodology used in CAMMS, ADMS, and Technical Annex. In case the terminology (e.g. notion of a building block) depends on deliverables of other WPs, it is referred to these deliverables. For presenting the assessment guidelines, the notion of a "target of assessment" is utilized (please see the List of Abbreviations and Glossary section). The following sections give a brief overview of CAMMS, ADMS, and other sources used CAMSS Among other methodologies, the starting point for this work will be the CAMSS methodology (Final Draft Revision of CAMSS, Version 1.0, March 2012). CAMSS is a method to assess formal specifications in the scope of egovernment. The deliverable is not restricted to CAMSS. The method is an initiative of the European Commission performed in close coordination with Member States. It allows for transparency in the choice of egovernment solutions and standards. However, whereas the CAMSS methodology is limited to specification and standards, the scope of WP3 is wider including broader issues of long term sustainability like governance and funding. Considering the limitations of the CAMSS, additional approaches must be defined to ensure the full business model for the sustainability of the building blocks and pilots are duly taken into account. Further holistic aspects have to be taken into consideration as described in the EIF and overall ISA activities, including the assessment of business models. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 16

17 In the CAMSS methodology, three scenarios are described. As the objective of the Guidelines is to propose a documentation of format and defining criteria for the maturity and sustainability assessment of building blocks, the CAMMS scenario "Assessment scenario 1 An assessment of a standardisation organisation" is not applicable to the e-sens project. Therefore in this document, the CAMSS scenarios "Assessment scenario 2 An assessment of a formal specification for adoption by public administrations" and "Assessment scenario 3 An assessment and selection of formal specifications for specific business needs and requirements" are taken into account. According to assessment scenario 2, a formal specification is assessed in order to evaluate and provide a recommendation on the possible adoption of the formal specification by the public administrations. This assessment scenario can be triggered by a public administration or related external stakeholders. The outcome for this assessment scenario will be the adoption of a certain formal specification by the public administrations. According to assessment scenario 3, a proposed set of formal specifications are assessed and evaluated in order to select and adopt the most relevant formal specification for specific business needs and requirements. If a certain business need arises and requires the adoption of a relevant formal specification, the business need should be examined to list the relevant requirements, and based upon those requirements, a selection of relevant formal specifications may be established. The outcome for this assessment scenario will be the selection and adoption of relevant formal specifications for the specific business needs and requirements. The basic difference between scenarios 2 and 3 is availability of specific business needs and requirements in scenario 3. In case such needs and requirements are not available, preference is given to scenario ADMS The CAMSS make use of the ADMS (Asset Description Metadata Schema) - a vocabulary to describe interoperability assets. Where possible, the current Guidelines use ADMS as well. In particular, the controlled vocabularies used during the assessment procedure are based on ADMS (Asset Description Metadata Schema. Specification Version 1.00). D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 17

18 1.3.3 Other sources Annex II of the Regulation on European standardisation 6 comprises requirements for the identification of ICT technical specifications. The JoinUp repository 7 currently covers the documentation of semantic and open-source software interoperability assets. The EFIR 8 contains interoperability assets of the Member States. 1.4 Relations to Internal e-sens Environment Relations to e-sens Work Packages WP3 has close connections with WP5 and WP6 both in development of the Guidelines and in doing the assessments. The collaboration activities with other WPs during developing the Guidelines are presented in Chapter 1.3 of the current document. Joint activities of the assessment procedure are depicted in Chapter 2. Task 3.1 will make it easier to focus on the generic, modular and re-usable character of building blocks being piloted in WP5 and further refined and consolidated in WP6. According to the Technical Annex (p 71), one of the Objectives of WP5 is to support WP3 in its mission to develop Long Term Sustainability activities for e-sens services and building blocks, by providing pilot-level and domain-level validation of requirements from a functional and organizational acceptance viewpoint. One of the tasks of WP6 deliverable D6.1 9 is taking stock - creating an ICT Architectural Baseline for e-sens (Technical Annex p 112). Each phase in the different WP6 Subgroups will have the main focus area of Inception. This focus area comprises assessment of ICT maturity and together with WP3 and WP5 - the overall maturity in accordance with the maturity model (Technical Annex p 114). e-sens Task 3.2 will use the Guidelines for the sustainability and maturity assessment. Task 3.4 will contribute to assessment of business need criteria. 6 REGULATION (EU) No 1025/2012 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 25 October 2012 on European standardisation. OJ L 316, , pp 12-33, 7 Joinup, 8 EFIR, 9 WP6 deliverable "D6.1 Executable ICT Baseline Architecture" D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 18

19 1.4.2 Reviewing and commenting on the Guidelines The current version of the Guidelines has taken into account comments on the earlier versions from contributors shown in Section 10 of the current document. The Guidelines are a living document that has to be updated after changes in underlying materials, due to experience gained in pilot assessments, as a result of new agreements, and other. 1.5 Relations to External e-sens Environment The Guidelines can be used for each architectural, solution and high level building block. Its novelty is that it goes beyond the state-of-the-art methodologies that focus solely on the assessment of standards and specifications. This deliverable makes it possible to focus on a set of standards and specifications. If the Guidelines are well accepted within the e-sens project, they will be published on joinup.ec.europa.eu. In this way they will be shared with a variety of stakeholders: institutional, national, private, industrial, judicial professions etc. This will enable these groups, and standardization organizations in particular, to profit from this knowledge and they are free to refine the methodology for their purposes. 1.6 Quality Management The overall quality of the development process of the Guidelines is based on the following. 1. Use of the most relevant, authoritative, and up-to-date frameworks and methodologies in development of the Guidelines. 2. Adherence to the requirements stated in the Technical Annex. 3. Adherence to the e-sens project templates, language requirements, and other quality specifications. 4. Staged development of the Guidelines comprising planning, preliminary draft, intermediate drafts, and the final version (see Chapter 1.3). 5. Coordination of the development activities with other WPs. 6. Reviews of the main principles and the drafts in collaboration with other WPs (the External Quality Team). 7. Discussions via and / or teleconference (be-weekly teleconferences of WP3 and Task 3.2, teleconferences of the WP3/WP5/WP6 joint Summer Task Force, private communications etc. 8. Keeping records of reviews and feedback. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 19

20 Remarks Checked by Conformance to e- SENS template OK (see item 3 above) Freek van Krevel, Anette Junikiewicz, Jaak Tepandi Language & Spelling OK (see item 3 above) Freek van Krevel, Anette Junikiewicz, Jaak Tepandi Delivered on time OK (see item 4 above) Freek van Krevel, Anette Junikiewicz Each technology description contains the correct elements Consistency with description in the TA and in other e-sens deliverables Contents is fit for purpose OK (see items 1-2,5-7 above) OK (see items 1-2,5-7 above) OK (see items 1-2,5-7 above) Authors, contributors Authors, contributors Authors, contributors Contents is fit for use OK (see items 1-2,5-7 above) Authors, contributors Commitment within WP OK (see items 1-2,5-8 above) Authors, contributors Table 3. Quality Management for the Guidelines 1.7 Risk Management The initial risk is that the deliverable is not accepted in the remainder of the project. Another risk is that the Guidelines are not followed - the core building blocks are being produced within one single silo and cannot be handed over properly to WP3 for a sustainability assessment. Mitigation measures for this should be made in Task 3.2, so that the Guidelines are being well accepted and have sufficient authority to be followed. Early awareness mechanisms and consensus building of the Guidelines prior to the sustainability assessment cycles in M18, M30 and M36 smoothen and accelerate the outcome of the sustainability assessment in itself. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 20

21 Description Probability Impact Priority Response Owner Non acceptance of the guidelines No proper followup by Task 3.2 No further adjustments to comply with experience gained in pilot assessments, result of new agreements, and other There are no resources for assessment of all BBs All the data for assessments are not available Likely High Top Further consensus building prior to assessments, escalation to MB or GA. Flexibility of the Guidelines Likely High Top Bringing point under the attention of Task leader Wp3.2. Guidelines as a supporting aid, not the mandatory list Likely Medium Medium Task leader WP 3.2 need to adapt Guidelines in close cooperation with Task leader WP3.1 Low Medium Medium Prioritization of BBs for assessment, planning for assessments, liaison with the e-sens project management Medium Low Medium The assessment criteria of the Guidelines can be modified. Flexibility of the assessment WP3 leader WP3 leader and Task leader WP3.2 Task leader WP 3.2 T3.1, T3.2, WP3, WP5, WP6, WP1 WP3, WP5, WP6, external stakeholders Table 4. Risk Management for the Guidelines D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 21

22 1.8 Legal Issues Not applicable within this deliverable. 1.9 Structure of the Document The document is structured as follows: Section 1 presents the introduction Section 2 - a procedure for assessment and an overview of the targets to be assessed Section 3 - the criteria for the proposal and consideration steps Section 4 is devoted to the assessment step and criteria Section 5 - to the recommendation step Section 6 deals with organizational aspects of assessment Section 7 presents the conclusions Sections 8 and 9 - references and contributors To help conducting future assessments, results of already existing assessments have been analysed to enable their identification and re-use. This part of the Guidelines is presented in Appendix 1, in order to enable the main users of the Guidelines (the experts performing specific assessments) to have fast access to the main body of the Guidelines (the evaluation procedure and criteria). Appendices 2, 3, and 4 provide additional information - proposal/information criteria based on CAMMS and on WP6 deliverable D6.1, as well as the CAMMS assessment criteria. The footnotes for the references are given at the beginning of each chapter and are not duplicated for other occurrences of the same reference throughout the chapter. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 22

23 2 The Assessment Procedure and Targets 2.1 The Assessment Procedure As presented in the introduction, WP3 has assessed applying CAMSS 10 scenarios as a means to develop a procedure for the assessment of building blocks. CAMSS scenarios 2 and 3, especially the assessment step (CAMSS, section 2.4) are considered for a basis for organizational aspects of assessment, together with relevant frameworks such as BOMOS2i 11, ISO/IEC , ISO/IEC series standards 13, and other sources as appropriate. The assessment is conducted by the e-sens WP3 Assessment Panel, providing transparency of the assessment process. Following the e-sens Technical Annex and the CAMSS methodology, the current document uses the assessment procedure that comprises proposal, consideration, assessment, and recommendation steps (Figure 1). 1. In the proposal step, a target of assessment (for example, an Architecture Building Block or High level Building Block, including support artifacts such as guidelines) is provided to Task 3.2 by WP6 Architectural Board. The target of assessment is provided using the proposal criteria. The proposal criteria provide general information about the proposed target of assessment, its status, items provided for assessment, and other. Some of the proposal criteria are highly important, the others are recommended. 2. In the consideration step, consideration criteria are used before the actual assessment, to validate information received and relevance of the proposal. 3. In the assessment step, the criteria used are categorised under standardisation, alignment with existing policy frameworks, and business need. Additional information will be sought from other Work Packages and external stakeholders. 4. In the recommendation step, recommendation criteria are applied to conclude with a classification (Discarded, Observed, Accepted, Recommended, and Mandatory) of the target 10 Common Assessment Method For Standards And Formal Specifications (CAMSS). Final Draft Revision of CAMSS. Version 1.0, March BOMOS2i, 12 ISO/IEC 12207:2008. Systems and software engineering. Software life cycle processes 13 ISO/IEC 25000:2005. Software Engineering. Software product Quality Requirements and Evaluation (SQuaRE). Guide to SQuaRE D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 23

24 of assessment. Depending on the e-sens mandate, it may be agreed to avoid the "Mandatory" recommendation in the assessments. This classification will be reported back to WP6 Architectural Board for using by other Work Packages. Figure 1 captures the primary process flow. One of the design principles for this document is flexibility of the assessment - Guidelines are intended as a supportive material to help with assessment. Assessments may be done iteratively, in cooperation with WP6, WP5, and other stakeholders. As an example, there are several options for timing piloting with respect to assessments. In some cases, it would be useful to have piloting results before the assessment and use these results in evaluating the criteria. From the other side, depending on the situation and decision of the Architectural Board, the assessment results may be used to decide about which building blocks to pilot. There can be many additional connections between the stakeholders. In particular, the Assessment Panel communicates with other Work Packages and external stakeholders during the assessment step. This communication may involve prioritizing the criteria used in assessment, considering additional criteria, evaluating the criteria, specifying procedures for the recommendation criteria, etc. Figure 2. The procedure for assessment with main stakeholders D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 24

25 2.2 The Targets of Assessment Since the main topic for the current Guidelines is documentation of format and defining criteria for assessment, only selected building block examples are mentioned below to improve understanding of the assessment issues. Further selection of building blocks will be based on input from WP6 during the assessment phase. The core building blocks for the first cycle of the sustainability assessment are as follows: e-id e-signatures e-documents e-delivery Other building blocks, such as Semantics, Circle of Trust can be added at a later stage as applicable. The targets of assessment are classified under broader categories of Architecture Building Blocks and Solution Building Blocks. The Architecture Building Blocks provided for assessment by WP6 may include but are not constrained to the following types. Architecture strategy, frameworks, and methodologies Service contracts, service level agreements, policies Documentation and guidelines, including software and deployment descriptions Specification of interfaces of components considering the building block as black box Specification of the functionality of the building block Detailed design of the building block and its internal components Other types It has been agreed in the management board of 2 July 2013 that Solution Building Blocks are beyond the targets of assessment for WP3. For the sake of completeness, the following types can also be regarded as targets for assessment, albeit for other purposes than in WP3. Implementation of the building block in terms of source files and its documentation Implementation of the building block in terms of executable files Other types These types may occur in combination. The complete list of building blocks and targets for assessment will be provided by WP6 Architectural Board. D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 25

26 To deal with different types of building blocks, the current guideline proposes a wide set of criteria that should be applied selectively as follows. Some criteria may be applicable to a specific type of building blocks The criteria may be highly important or recommended There can be assessment levels defined for the criteria D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 26

27 3 Proposal and Consideration Steps: Documentation of Format The documentation of formats provides two sets of information/criteria. First, it describes the proposal information/criteria used by WP6 to provide building blocks for assessment. Second, it describes consideration criteria used by Task T3.2 to preliminarily evaluate the proposal. The documentation of formats is based on several sources: CAMSS 14 sections 3.1 and 3.2 deliverable "D6.1 Executable ICT Baseline Architecture" Asset Description Metadata Schema 15 (ADMS) other relevant standardization and asset description initiatives such as the Regulation on European standardisation 16, the JoinUp platform 17, the European Federated Interoperability Repository 18 (EFIR), and others. 3.1 The Proposal Step As input to the proposal step, a target of assessment is provided by WP6 Architectural Board. The target of assessment is provided using the proposal information/criteria. In the e-sens reference materials, the following sets of proposal information/criteria are available. Proposal information/criteria based on CAMMS Section 3.1 and the ADMS specification v 0.8. The main categories of criteria during the proposal step comprise asset description and relationship. For reference, the CAMMS information/criteria are given in Appendix 2. Proposal information/criteria based on WP6 deliverable "D6.1 Executable ICT Baseline Architecture" (section 2.5, "Building Block Evaluation Model"). For reference, the D6.1 information/criteria are given in Appendix Common Assessment Method For Standards And Formal Specifications (CAMSS). Final Draft Revision of CAMSS. Version 1.0, March ADMS. Asset Description Metadata Schema. Specification Version Release date 18/04/ REGULATION (EU) No 1025/2012 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 25 October 2012 on European standardisation. OJ L 316, , pp 12-33, 17 JoinUp, 18 EFIR, D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 27

28 Proposal information/criteria based on the ADMS concept "Semantic Asset" (section of the document "ADMS. Asset Description Metadata Schema. Specification Version Release date 18/04/2012"). In case of a software component, the proposal information/criteria may be based on the ADMS.SW 19 main concepts: software project, software release, software package, software repository (section 4.2 of the document "Asset Description Metadata Schema for Software ADMS.SW Release date 20/05/2012"). As WP3 will not evaluate SBB, this set is here mentioned for the sake of completeness and in case of possible subsequent modifications. These proposal information/criteria are integrated into one set according to two principles: (1) merging the information/criteria from CAMMS, ADMS, and the D6.1 deliverable and (2) keeping ADMS as the central model for the merge. The following sub-sections include (1) the complete list of proposal information/criteria and (2) for convenience, the highly important information/criteria in the recommended order of presentation. Terminological note. Different sources refer differently to the proposal items: D6.1 - as "information and criteria"; ADMS - as "property" and "relationship"; CAMMS - as "criteria". In the current document, the proposal items are referred to as "information/criteria" Complete list of proposal information/criteria The complete list for the proposal information/criteria given in the table below comprises the following fields. Nr - category number. - CAMSS category or D6.1. Description - category description. Reference - reference for the information/criteria (CAMSS number, ADMS, or D6.1). Note: only CAMMS information/criteria that are included in the ADMS v 1.0 are given. Information/criteria - name of the information/criteria. Description - description of the information/criteria, or reference to source information. H/R - highly important / recommended. The table indicates only highly important (H) items; all other information/criteria are recommended. Highly important information/criteria are: (1) those that are used to Identify an asset according to the table in ADMS Section 4; (2) 19 Asset Description Metadata Schema for Software ADMS.SW Release date 20/05/2012, D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 28

29 those that are included in the deliverable D6.1, but are not assessed by WP3. Note: in case of items assessed by WP3, there can still be a preliminary optional assessment by WP6 included in the proposal. Nr Description 1 CAMMS Providing asset the description general infor- mation for the proposed TOA Reference CAMSS P.1 CAMSS P.2 CAMSS P.3 CAMSS P.4 CAMSS P.5 Information/ criteria Alternative name Date of creation Date of last modification Description ID Description Alternative name for the asset. Note: this information may be used to provide additional access points, e.g. allowing indexing of any acronyms, nicknames, shorthand notations or other identifying information under which a user might expect to find the asset Creation date of this version of the asset Date of latest update of asset Descriptive text for the asset, if applicable including its purpose and functions URI for the asset (a globally unique, but not always humanreadable identifier) ADMS Identifier Any identifier for the asset (a human-readable identifier) CAMSS Keyword Word or phrase to describe the P.6 asset ADMS Metadata date Date of the most recent update of the metadata for the Asset CAMSS Name Name of the asset. See also D6.1 P.7 Section CAMSS Version Version number or other P.8 designation of the asset ADMS Format Format in which an asset is distributed (e.g. PDF, XSD, RDF/XML, HTML, ZIP). See also ADMS Section 6.3 ADMS Version notes Description of changes between this version and the previous version of the Asset H/R H H H H H H H H D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 29

30 Nr Description 2 CAMMS Providing relationship the informa- tion on the status of the specific TOA being proposed Reference CAMSS P.9, D6.1 Information/ criteria Asset type Description Classification of an asset according to a controlled vocabulary, e.g. code list, metadata schema. See also ADMS Section 6.2, D 6.1 Section ADMS Contact point Contact point for further information about an asset CAMSS P.10 Current version Current or latest version of the asset ADMS Home page A Web page that is fully dedicated to the asset ADMS Included asset An asset that is contained in the asset being described, e.g. when there are several vocabularies defined in a single document ADMS Included item item that is contained in the asset (e.g. a concept in a controlled vocabulary, an individual code in a code list or any other atomic element) CAMSS P.13 Interoperability level Level according to the European Interoperability Framework (EIF CAMSS P.14 CAMSS P.11 Language Main documentation 2.0) for which an asset is relevant Language of an asset if it contains textual information, e.g. the language of the terms in a controlled vocabulary or the language that a specification is written in The main documentation or specification of the asset ADMS Metadata language Language of the metadata for the asset ADMS Metadata publisher Organisation making the metadata for the asset available ADMS Next version Newer version of the asset CAMSS P.15 Previous version Older version of the asset H/R H H D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 30

31 3 D6.1 Items from D6.1 not included above Nr Description Reference CAMSS P.16 Information/ criteria Publisher Description Organisation making the asset available ADMS Related asset Assets related to the asset ADMS Related documentation Documentation that contains information related to the asset ADMS Related web page A Web page that contains information related to the asset ADMS Distribution Implementation of the asset in a particular format ADMS Repository origin Repository that contains the primary description of the asset ADMS Sample Sample of the asset CAMSS P.18 CAMSS P.20 Spatial coverage Status Geographic region or jurisdiction to which the asset applies Status of the asset (Completed, Under development, Deprecated, Withdrawn), see ADMS sections and 6.10 ADMS Temporal coverage Time period relevant to the asset, e.g. its validity ADMS Theme Theme or sector to which the asset applies ADMS Translation Translation of the asset D6.1 Relationship to ICT Strategies in EC/MS See D6.1 section D6.1 Requirements See D6.1 section H D6.1 Generic/Specific See D6.1 section H D6.1 Architecture See D6.1 section Framework / Component Specifications D6.1 Standards See D6.1 section D6.1 Open Source See D6.1 section H D6.1 Relationship and Couplings See D6.1 section H/R H H H D3.1 Guidelines to the assessment the sustainability and maturity of building blocks 31