Healthcare Interoperability Testing and Conformance Harmonisation

Transcription

1 Healthcare Interoperability Testing and Conformance Harmonisation WP6: D6.4 Final Report including consolidated Roadmap and recommendations Version July 2011

2 DISCLAIMER Possible inaccuracies of information are under the responsibility of the project/study team. This report reflects solely the views of its authors. The European Commission is not liable for any use that may be made of the information contained therein. D6.4 Final Report and Roadmap Page 2 of 36

3 EXECUTIVE SUMMARY This Final Report provides an overview of the outcome of the Healthcare Interoperability Testing and Conformance Harmonisation (HITCH) Project. This report includes the project s recommendations and a proposed roadmap for achieving sustainable and effective deployment of the testing and certification/labelling process to enable health information exchange interoperability within and between European member states. As stated in the Calliope Interoperability Roadmap: Process improvements, acceleration of testing, and adoption and use of cross-sector standards are being pursued through close co-operation with national, European and international standardisation organisations. This in turn strengthens and binds together the European health IT industry and stimulates partnerships. The immediate benefits for the public sector and health authorities mean that they can begin to rely on open and sustainable IT solutions. HITCH is focused on one of the most critical elements in the realisation of ehealth interoperability: the acceleration of testing for interoperability, and supporting processes. The Project focuses on how to test for conformance with profiles and standards that have already been selected. Its ambition is to help answer questions such as: How can an IT system be tested for conformance with agreed standards and profiles? Are the necessary and reliable interoperability test tools available to validate? Can there be a way to declare conformance, such as a label or a certification process? Will it be possible to test a specific product against others to validate their interoperability? What needs to be organised at the European and national/regional levels to test and certify interoperability, avoiding duplicate efforts and accommodating national specificities? Addressing these questions is of critical interest to various groups: Vendors, Users, Patients and Authorities, each with specific expectations. The aim of the HITCH project was to involve, both within the project and through reviews, those stakeholders who are at the heart of Interoperability issues in defining and agreeing on a roadmap to establish a way forward for Interoperability Conformance Testing in the field of Healthcare. HITCH offers an analysis of the state of the art in quality systems governing interoperability, the maturity of test tools and various alternatives in certification and labelling processes. Based on this assessment, six recommendations were identified as critical to moving forward: 1. Drive adoption of interoperability standards and profiles at European level This is a critical precondition to enabling the development of consistent test tools and certification/labelling. Profiles recognised at EU-level would reduce the fragmentation of ITsystems and the associated cost of development and maintenance. If recognised profiles D6.4 Final Report and Roadmap Page 3 of 36

4 have the necessary flexibility, such as IHE profiles, they could allow for national/regional customisation. Such recognised profiles are expected to form the technical and semantic dimension of an upcoming EU Interoperability Framework. 2. Drive adoption of the HITCH Interoperability Quality Management System Guideline. Such a specific interoperability testing Guideline would define policies and procedures for managing the quality of profile testing based upon well-established standards (e.g. ISO 9000 and ISO 17025). It would be used for laboratory testing and inspection bodies in the field of interoperability testing. The HITCH recommendation is to encourage the use of its proposed Quality Guide by entities such as IHE in organising the epsos Projectathon and its Connectathon. Based on feedback, this draft guide should then be fully developed into an international standard. 3. Drive closure of key gaps in interoperability testing tools Testing tools are essential for many aspects of smooth testing processes, critical to accelerating implementation and deployment of interoperability. The use of tools increases productivity in test execution, reliability of the results obtained, traceability of test evolution and repeatability of tests. The evaluation of the Open Source test tools currently available has demonstrated that key gaps need to be addressed: Terminology services; Devices workflow; Exchange and sharing of electronic records (ehealth; einclusion); Identification, authentication and signature; Structured documents (laboratory, oncology and other specialities). Improvement of test management tools and languages (e.g. TTCN-3 test plans, IHE Gazelle) to increase the level of test automation wherever feasible 4. Encourage collaboration between the functionality-focused and interoperability-focused testing/labelling initiatives Implementations may be tested for: Interoperability: this ensures that information exchange meets the standards-based profiles Functionality: this ensures that the features expected are provided to the user of the implementation As one may choose to test from only one or both of these points of view, they need to be distinguished from but also related to each other as combined test plans may be easier to test in some cases. D6.4 Final Report and Roadmap Page 4 of 36

5 One of the conclusions of the work conducted in HITCH by IHE (Interoperability testing) and EuroRec (Functionality testing) is that it is best to keep the test scripts separate for addressing functional criteria or interoperability criteria. However, a link between those criteria exists and need to be documented. 5. Preserve flexibility across the proposed Labelling and Certification schemes Various approaches to certification and labelling processes have been analysed by HITCH. Three typical alternatives have been identified as most relevant to ehealth interoperability: Labelling/Certification of products by a Third Party: The testing process is performed by a conformity assessment body (accredited third party) or a (non-accredited) third party according to the specifications and requirements within the scope of the certification. This can be seen as an a priori control. Self-assessment of products with external Audit: The testing process is performed by the supplier (like self-assessment) according to the specifications and requirements (that represent the scope of the labelling/certification processes) and within an agreed interoperability quality assurance framework. The vendor is subject to an external audit performed by an inspection body (accredited or non-accredited third party). This can be seen as an ex-post control. Self-assessment of products: The testing process is performed by the vendor according to the testing specifications and requirements (that represent the scope) and within an agreed interoperability quality assurance framework. Each of these alternatives has benefits and limitations. The choice depends on: The maturity of the market; The cost of product development and administrative processes; The scale of diffusion of interoperability and the emergence of new interoperability profiles; The legal framework relating to patient safety and security of exchanged or shared medical data. In a less mature or emerging market, Third Party Labelling/Certification of products seems to be a reasonable first step approach to developing an initial range of interoperable products. As the market becomes more stable and mature, there should be a natural transition to selfassessment with external audit as it is less cumbersome and more dynamic. The broad acceptance of the specifications and their adoption by suppliers will be encouraged by this scheme. HITCH recommends a flexible approach to the choice of Labelling/Certification scheme. It is advised not to include any choice of a specific certification/labelling scheme in a regulation or law to allow progressive transition towards lighter and equally effective schemes whenever possible. This is important to reduce costs and increase market adoption while accompanying the maturation of the market. D6.4 Final Report and Roadmap Page 5 of 36

6 6. Establish a two-level Labelling and Certification process (European Level and National/project Level) At the European Level At European level, it is proposed to limit the focus of testing and certification to the EUrecognised profiles (see recommendation 1). This foundation will create a common level of consistency and quality in ehealth interoperability, a critical step towards developing a Europe-wide market for ehealth interoperability. It is proposed to develop a testing ecosystem by building and evolving the European IHE testing process: first by implementing the HITCH QMS Guideline, enhancing and aligning the test tools on the EU-recognised profiles, enhancing the quality of the test tools to become a reference as they have been validated by the Connectathon itself and eventually moving from the current labelling approach to a certification scheme. An outcome of this evolutionary approach is to recognise the test plans, test cases, test tools and test management tool (Gazelle) whilst openly offering them for the self-assessment of products, either directly by the projects or at country level testing (see below). At the Project/Country level ehealth projects may be led by public or private organisations: European projects such as epsos; National projects such as DMP in France, or ELGA in Austria; Regional projects such as Veneto and Abruzzo in Italy, or the regional PACS deployment in Wales; Other projects promoted by a given user category (hospital, health professional network, etc.) The proposed certification/labelling and testing strategy applies to projects that have decided to leverage the European-recognised profiles within some local extensions (additional specific profiles or extensions of the existing profiles). As the European-recognised profiles have an established corpus of test plans, test cases and test tools, it will be to the benefit of the project/national level to reuse these valuable resources along with their own extensions. This will ensure that consistency between European and national levels is preserved and that additional project/national level investment and testing is focused only on the project/national extensions both for the project and the vendors. By using the same testing processes based on the same Quality Management System used at European level or by accredited bodies, shared experience and cross-recognition will be possible. Proposed Roadmap The HITCH project proposes a roadmap in order to orchestrate the implementation of the above recommendations. This roadmap spans four years and is organised into three phases: 1. Get ready (could in fact start immediately) This first phase delivers the European-Level Test Process and Certification/Labelling Process. It is assumed that the first version of the ehealth D6.4 Final Report and Roadmap Page 6 of 36

7 Interoperability Framework becomes available at the end of this phase, as it is a critical prerequisite for the next phase. 2. Operationalise The second phase is to perform an evaluation of the draft European testing and labelling process and the first version of the test tools developed in the previous phase. Lessons learned from this evaluation are to be used to revise and formally approve the European testing and certification process and deliver a new version of the testing tools. 3. Deploy 2015 This third pilot phase offers testing and certification of ehealth products. The approved European testing and certification process is applied using the second version of the test tools delivered by the previous phase. For each of these phases, several detailed milestones have been defined for the underlying testing and certification processes (Labelling/Certification process, European QMS) as well as for the underlying test tool development (test management platform, test cases and test tools) (see complete Roadmap on page 36). D6.4 Final Report and Roadmap Page 7 of 36

8 DOCUMENT COVER AND CONTROL PAGE Project number: FP7-ICT Project name: HITCH. Healthcare Interoperability Testing and Conformance Harmonisation Document id: D6.4 Document name: Tools Selection Dissemination level* PU: Public Version: 1.0 Date: Author(s): Karima Bourquard Eric Poiseau Contributors *Dissemination level: PU = Public, PP = Restricted to other programme participants, RE = Restricted to a group specified by the consortium, CO = Confidential, only for members of the consortium. ABSTRACT This task aims to consolidate the results of the project. Based on the inputs generated by the HITCH work packages, complemented with the feedback from the major stakeholders attending the IHE European Connectathon (vendors, users, patients and Member States and European authorities), the HITCH roadmap will propose an achievable foundation for Interoperability Conformance Testing with deployment starting in The roadmap will identify specific needs in terms of strategy, process improvements and tool development. Recommendations and identified deployment activities are presented in three phases from 2012 to KEYWORDS Interoperability, requirements, conformance testing, roadmap, ehealth D6.4 Final Report and Roadmap Page 8 of 36

9 TABLE OF CONTENTS EXECUTIVE SUMMARY... 2! 1! ABOUT THIS DOCUMENT... 11! 1.1! Acknowledgments... 11! 1.2! HITCH Team... 11! 2! INTRODUCTION... 11! 2.1! Overview... 11! 2.2! Rationale... 12! 3! OVERVIEW OF STATE OF THE ART (WHERE WE ARE TODAY)... 14! 3.1! Quality Management System (QMS)... 14! 3.2! Tools... 15! 3.3! Certification... 18! 4! VISION FOR THE NEAR FUTURE... 21! 4.1! Quality Management System (QMS)... 21! 4.2! Test Tools... 23! 4.3! Certification and Labelling Process... 25! 5! RECOMMENDATIONS... 26! 5.1! Drive the Adoption of Profiles at European Level... 26! 5.2! Drive adoption of the HITCH QMS... 27! 5.3! Drive Closure of key Gaps in Interoperability Testing Tools ! 5.4! Encourage Collaboration between Functional- and Interoperability-focused Initiatives 28! 5.5! Preserve Flexibility across LabelCert Schemes... 29! 5.6! Establish a Two-level LabelCert Process... 30! 6! ROADMAP... 31! 6.1! Get ready: ! D6.4 Final Report and Roadmap Page 9 of 36

10 6.2! Operationalise: ! 6.3! Deploy: 2015 and beyond... 34! D6.4 Final Report and Roadmap Page 10 of 36

11 1 ABOUT THIS DOCUMENT 1.1 Acknowledgments We thank our colleagues from the European Commission s ICT for Health unit for their kind support, in particular our project officer Benoit Abeloos for his continuous support throughout the realisation of this project. The HITCH team is grateful to the speakers and participants at the workshop in Pisa during the IHE European Connectathon 2011, in particular Lisa Carnahan from the NIST and Benoit Abeloos. 1.2 HITCH Team This project is realised by partners already at the heart of ehealth testing: IHE Europe and EuroRec from Belgium, INRIA and ETSI from France as well as MedCom from Denmark and OFFIS from Germany. This document is the final outcome of work supported by the European Commission, Directorate Information Society, through the EU Seventh Framework Programme (FP7). The HITCH project was executed under Support Action contract number FP INTRODUCTION 2.1 Overview Today, delivering healthcare to patients includes a multitude of computer systems that collaborate with each other by exchanging patient data. This is necessary as many different devices and departments as well as local, regional and national institutions are engaged in the treatment of a patient. At European level, the directive on patients' rights in cross-border healthcare aims to establish rules for facilitating the access to safe and high quality cross-border healthcare in the Community and to ensure patients mobility in accordance with the principles established by the Court of Justice and to promote cooperation on healthcare between Member States. To promote and test this coordination, the epsos project is experimenting with exchanges of medical data between countries by specifying a technical framework based on integration profiles and standards. The European Calliope project has clearly stated that Process improvements, acceleration of testing, and adoption and use of cross-sector standards are being pursued through close co-operation with national, European and international standardisation organisations. This in turn strengthens and binds together the European health IT industry and stimulates partnerships. The immediate benefits for the public sector and health authorities mean that they can begin to rely on open and sustainable IT solutions. HITCH is focused on one of the most critical elements in the realisation of ehealth interoperability: the acceleration of testing for interoperability, and supporting processes. D6.4 Final Report and Roadmap Page 11 of 36

12 2.2 Rationale All collected information such as medical images (X-Rays, ultrasound, etc.), and medical reports need to be available to the patient's care providers in order to facilitate optimal treatment and at the same time avoid redundant examinations and adverse drug reactions. Furthermore, the current trend of having all patient-related data ubiquitously available (e.g. in the patient's holiday resort) requires a high level of interaction between the computer systems involved. The underlying communication of images, reports and other information is highly complex and needs to be supported by international standards including terminologies to express semantics such as medical terms (for example, terminology for describing anatomical regions, diseases, physical units, medication, pharmaceutics and many more). The final goal would be to have all information shared such that a computer system may "understand" and evaluate it. Furthermore, the systems should be able to provide an understandable presentation to the user utilising the system, e.g. to a physician or the patient themselves. This only works if all involved systems communicate in an internationally designed, familiar and standardised way. If two systems can talk to each other, then there is "interoperability" between those systems. According to the IEEE Standard Computer Dictionary, interoperability can be defined as the ability of two or more systems or components to exchange data and to use the information that has been exchanged. In addition, in the field of ehealth the definition can be extended to include the ability of two or more systems to exchange data: between linguistically and culturally disparate health professionals, patients and other actors and organisations; within and across health system jurisdictions in a collaborative manner. Interoperability must first be addressed by the conformance harmonisation of ehealth systems. This is sometimes also referred to as profiling, i.e. selecting (subsets of) standards and options in order to uniquely identify the communication means to be used. As described, these standards are fairly complex, for example the DICOM standard describing the content and communication of medical images spans over 4,000 pages of technical documentation. So, besides describing how the communication should work, it has to be Profile A profile is a selection of definitions and options from existing standards. Profiling is usually conducted in order to achieve interoperability between different products and implementations as a profile aims to harmonise all systems implementing it to use the same standards and contents. Interoperability The ability of two or more systems or components to exchange information and to use the information that has been exchanged. (IEEE 610) ensured that the different software products actually do what is defined in the standards. In this context, two different testing methods are known. On the one hand, testing whether a system conforms to a set of standards is called conformance testing. On the other hand, interoperability testing "only" tests whether two systems are able to exchange information with each other 1. Interoperability Conformance testing is a 1 i.e. aims to test whether the communication goal is reached regardless of whether both are also conformant to the standards they implement D6.4 Final Report and Roadmap Page 12 of 36

13 topic of interest to various groups: Vendors, Users, Patients and Authorities all have their views on the subject. Vendors wish to avoid developing interfaces for information exchange specific to each site. Users expect products to interoperate smoothly with minimal integration effort, even when the products are provided by different vendors. For health professionals, conformance testing is critical to ensuring quality transfer of clinical information and enabling a safe and secured environment. Patients care about their safety, and would like to receive timely and effective care avoiding redundant examinations. Regional and national health authorities are concerned with the quality of care and its costs. Interoperability Conformance Testing reduces the deployment costs and risks of multisystem IT projects. It results in projects that have a more predictable deployment schedule and reduced disruption to existing care delivery activities. The aim of the HITCH (Healthcare Interoperability Testing and Conformance Harmonisation) project was to involve major stakeholders already at the heart of Interoperability issues in defining and agreeing on a roadmap to establish a foundation for the Interoperability Conformance Testing in the field of Healthcare. Three main topics have been identified in order to obtain a credible and applicable testing process: The ehealth Interoperability Testing Quality Management System (QMS) which describes policies and procedures for managing the quality of profile testing; ehealth Test Tool analysis which describes the current situation with regard to available Open Source test tools, the gaps and needs for the future; The strategy for labelling and certification of ehealth products in Europe and, by extension, the links between projects and national programme level on the one hand and the international level on the other. In the following sections, the state of the art of each of the above topics is discussed in further detail, followed by the vision of the near future. This report concludes with a set of recommendations and a proposed roadmap for achieving sustainable and effective deployment of the testing and certification/labelling process to enable health information exchange interoperability within and between European member states. This roadmap complements the other roadmaps proposed by other European projects such as Smart Personal Health, CALLIOPE, EHR-Q, etc. and more generally could contribute to the deployment of the European Cross Border Health Care directive. D6.4 Final Report and Roadmap Page 13 of 36

14 3 OVERVIEW OF STATE OF THE ART (WHERE WE ARE TODAY) 3.1 Quality Management System (QMS) The main thrust of a QMS for Interoperability Testing is in defining the processes, which will result in the production of quality products and services, rather than in detecting defective products or services after they have been produced. Developing and implementing a QMS is not a new discipline and has been conducted in industry (e.g. construction and car production) for many years. The search for projects and experiences regarding QMS for A QMS can be defined as a set of coordinated activities to direct and control the work in an organisation in order to continually improve the effectiveness and efficiency of its performance 2. Interoperability Testing did not detect much useful material. As a consequence, it was necessary to gather and analyse information (Figure 1) that could form a common understanding defining the QMS requirements for Interoperability Testing.!"!"#$%&' (#)#*+,+)& -'.&+, / #" 45#$"#&%6)167 /)&+869+8#:%$%&' %)%&%#&%5+. $; 4<%.&%)* %)&+869+8#:%$%&' #)=1 >6)768,#)>+ /)%&%#&%5+. '" 4D 96$%>%+.18+$#&+=1 &6 %)&+869+8#:%$%&' %"?+7%)%&%6)1671&@+1 8+A"%8+,+)&.B!(-1 %)&+869+8#:%$%&' &+.&%)* &" -&#C+@6$=+8 8+A"%8+,+)&. Figure 1: Methodology for defining QMS requirements A number of interesting existing interoperability initiatives have been identified and evaluated based on the eight interoperability QMS principles defined in ISO The list of reviewed initiatives was not exhaustive and served as input to the interoperability QMS requirements. The reviewed initiatives were CCHIT, Continua, ETSI, EuroRec, IHE, Infoway, KITH, MedCom and OpenECG. 2 D6.4 Final Report and Roadmap Page 14 of 36

15 Score Optimized 5 Managed 4 Defined 3 Focus in HITCH Principle 4 & 5. Repeatable 2 Initial 1 Not existing Interoperability QMS principle Figure 2: Outcome of the evaluation of interoperability initiatives The evaluation led to the following conclusions for the existing interoperability initiatives: In general they have good knowledge regarding the ehealth domain and are involved in the process of supporting and improving interoperability, including the necessary stakeholders (principles 1-3 and 8) in that process. Quality is not monitored systematically and decisionmaking could be improved (principles 6-7). They lack formalised procedures and governance structures dealing with qualityrelated matters (principles 4-5). 3.2 Tools The eight interoperability QMS principles: 1. Customer focus 2. Leadership 3. Involvement of people 4. Process approach 5. System approach to management 6. Continual improvement 7. Factual approach to decision making 8. Mutually beneficial supplier relationships Reasons for interoperability problems are always related to a lack of standards, ambiguous standards or the incorrect interpretation of standards. To some extent, interoperability problems can be identified and resolved by testing systems before their deployment. Test tools are an important factor in conformance and interoperability testing and the HITCH project therefore reviewed over 30 tools and libraries of interest in the context of interoperability conformance testing of healthcare information systems. To this end, an evaluation approach was established based on the ISO PECA methodology. It proposed a detailed set of evaluation criteria ranging from name, version, author and programming language to quality management measures such as bug management and public source code repositories. The review revealed a rich range of test tools and libraries, many of them freely available under a non-restrictive Open Source licence. The detailed evaluation of each tool provides a strong support to users such as vendors, authorities, certifiers, service technicians and others interested in ehealth interoperability testing in selecting the testing tools best suited to their specific requirements. For each tool under review, a radar plot-like visualisation was D6.4 Final Report and Roadmap Page 15 of 36

16 created, giving a visual illustration of the tool s usefulness and serving as a quality indication (Figure 3). Each individual evaluation was based as far as possible on objective measures. However, in order to provide helpful indications of a tool s usefulness, some subjective influences cannot (and should not) be avoided, but are documented where applicable. The HITCH evaluation team members ehealth testing experience reflected in some ratings is even seen as a valuable addition to the tool facts that were collected from documentation and, in rare cases, source code.! Figure 3: Example of Radar Plot (NIST PIX/PDQ Pre-Connectathon Test Tool) A common task for a vendor implementing a specific product or a certifier testing those products is to find the correct testing tool to check the interoperability of the implemented software. Besides the rich tool descriptions offered within deliverable D2.1 3, the evaluation provided an overview of the most common test areas such as HL7, DICOM and test management. For these domains, Domain Maps were introduced showing all the relevant tools of a specific domain at a glance. The tools within a domain map are displayed in a coordinate system offering rapid understanding of whether a tool is more generic or specialised and whether it works more on a high application level or on the technical protocol side. 3 HITCH Deliverable 2.1 D6.4 Final Report and Roadmap Page 16 of 36

17 Functional/applicative Gazelle TestLink specialized Eurorec Use Tools TM generic Selenium TTCN3 based Test Management tools SoapUI!"#$%&#'()*+,-,#,(. Test Management Figure 4: Example of the domain map for tools in the field of test management. As a high level overview, deliverable D2.1 also presented a tool landscape visualisation (Figure 4) which brought together all evaluated tools in a single figure that organises the world of ehealth testing tools into different areas (test management, privacy, physical transport, etc). The figure provides a quick overview of how many tools exist for each area. Against this background, it is noted that Certification Based on ISO 9001:2000 (or ISO 9001:2008) and ISO 14001:2004, certification could be defined as an independent accredited external body issuing written assurance (the certificate ) that it has audited and verified that the product or software conforms to the specified requirements. the tool support for test management tools and tools for processes and methodologies as well as health information exchange infrastructure, seem to be thin if looking at the number and completeness of the tools available. In general, the review revealed trends towards building large-scale ehealth infrastructure tools with projects such as Open Health Tools (OHT) or the Open ehealth Integration Platform. As a result, from the different levels reviewed (detailed tool information, radar plot, domain maps and tool landscape), the following statements can also be derived: As the maturity of the tools increases, the quality management established for each tool increases. The more mature a standard or specification is, the higher the quality of tools for testing it. D6.4 Final Report and Roadmap Page 17 of 36

18 3.3 Certification In contrast to the US, where the certification approach is defined at federal regulation level, the situation in Europe differs and depends on the individual European countries. Figure 5 below has been designed based on the definition of the certification and labelling procedures identified in different European countries. The purpose of the figure is to describe the approach commonly used in certification. It provides an overview of the relations between relevant bodies (e.g. entity requesting certification, accredited entity and testing or auditing entities) and the tasks of each of these organisations within the certification process. Audit An audit is an independent, objective assurance and consulting activity designed to add value and improve an organisation s operations. It helps an organisation accomplish its objectives by bringing a systematic, disciplined approach to evaluating and improving the effectiveness of risk management, control, and governance processes. Based on the definition of The Institute of Internal Auditors on June 29 th D6.4 Final Report and Roadmap Page 18 of 36

19 Certification Process International (IAF), European (EA) or national Accreditation forum/organisation International Accreditation Forum European co-operation for accreditation Member State (coordination of the activities of the accreditation bodies) Accreditation Authority Notifies Conformity Assessment Bodies (CABs) Assess Products including services Supplier Third party assess the competence of CAB who is conform to a define scope of activities Testing Laboratory Inspection Body Conformity of products, services and suppliers to requirements and criteria Derived from ISO 17011:2004 Figure 5: Certification Process Certification processes exist in many domains 4 (security, automotive, banking and others) and in general the following is observed for these processes. The accreditation authority is a European/National entity that conforms to the ISO and/or guide 65 (for example, COFRAC in France). The accreditation authority has the competence to accredit inspection or laboratory bodies. This accreditation authority can be a member of a certification body network where the activities of such entities are coordinated. The network may be established at European or international level. The network gives the accreditation authorities recognition of their competency. For example, the multilateral agreement (MLA) between accreditors provides a means for goods and services to cross boundaries in Europe and throughout the world (example: IAF- Certified once, accepted everywhere ). To become a conformity assessment body, an organisation needs to conform to requirements based on technical competence, impartiality, availability of skills and equipment, professional privacy protection and civil liability insurance. The certification is based on the reliability of and confidence in the competence of the body performing the certification. However, a laboratory or inspection body may not be certified but nonetheless provide a high level of confidence due to its dedication to meeting the necessary requirements in fields where no CABs exist. To build a certification process in a new field takes time and is a long process that depends on: Definition of the specifications and requirements; Definition of the requirements for the laboratory and/or inspection bodies; 4 D6.4 Final Report and Roadmap Page 19 of 36

20 Setting up a certification process; Developing competence, skills and equipment; Etc. In the ehealth interoperability sector, the first attempts at starting a process leading to recognition that products conform to specifications have been made by bodies such as IHE and Continua Alliance, as well as EuroRec for functional aspects. These bodies are not accredited and the labels they deliver are not fully recognised by the whole market for the following reasons: The specifications and requirements are recently defined and still need to be improved; The processes, tools and equipment needed to launch product certification are in a developmental stage. Considering that the labelling and certification processes are very similar and that the difference is a question of the authority i.e. organisation that has legal and or regulatory authority to perform labelling or certification, we decided to introduce the term LabelCert process in this document. The distinction between the LabelCert scenarios (using the LabelCert process) is based on the following principles: Bodies participating in the LabelCert process: CAB (for certification), testing laboratory body (e.g. IHE) or inspection body (e.g. EuroRec for functional aspects) and participants who present products and their own testing processes. Note that some CABs are able to support both activities (inspection and testing laboratory). The scope of the activities of the LabelCert scenarios focuses primarily on functional auditing and testing of both conformance and interoperability. In addition, in the case of self-assessment with external audit, it also includes auditing the testing process as described in the QMS. Figure 6 shows the generic LabelCert process a supplier has to follow in order to obtain his seal. D6.4 Final Report and Roadmap Page 20 of 36

21 Requirements definition V Signature of agreement with the software company V Submission of the logs to the NO/AC/TO V Audit and/or test Results and agreement validations V Software Certified Publication of the certified software list Figure 6: Generic LabelCert processes The requirements, specifications and test criteria are provided by the organisation that sets up a label/certificate for products and services. The supplier signs an agreement with this organisation or, by delegation, with the accredited authority (called CAB). Once the process starts, the supplier submits a log (test results and/or documentation) to the CAB. After validation of the input data, an inspection or lab testing is launched. Following validation, the label/certificate is given to the product or testing services and published. 4 VISION FOR THE NEAR FUTURE Based on the state of the art described above, this chapter describes the desired future of interoperability testing in Europe, which it is believed can be reached within the next five years. The vision is presented for the three areas addressed by HITCH: QMS, testing tools and the LabelCert process. 4.1 Quality Management System (QMS) HITCH presented 5 how a Quality Management System for interoperability can be set up, based on principles taken from ISO The vision is for the current interoperability initiatives organised by parties such as IHE and EuroRec to achieve an initial implementation of the HITCH-defined Interoperability Testing QMS in the next years. Eventually, the QMS could be based on a new international standard dealing specifically with this topic. A group of experts from Europe and the US will extend the first version of the HITCH QMS to a Quality Guide for laboratory testing and inspection bodies in the field of interoperability 5 HITCH Deliverable 1.1 and Deliverable 2.2 D6.4 Final Report and Roadmap Page 21 of 36

22 testing. The Quality Guide could then be further developed into an international standard by ISO. In the same timeframe, IHE and EuroRec (at least) should instantiate the necessary QMS processes (see Figure 7) and actors as shown in Table 1. &2.+'"%31"4,+4(-.)%:"0,";% *"#,/)%$"#$#% :"<9,2"=")$#% &'(')" *"($+%,-.'$"%%!"#$% &'(')" *"($% /0'12$3% 41'((2()% %!"#$% &'()% *"+),-.)% % *"0"'.1%.2%3"'"4$%%!"#$%!..'#% 5(',6($"%!"#$%D(#"#%:"0,";% 3,=9'($.2#E5(',6($.2#E342,1$#E*($(% F")"2($.2#% *.49=")$(-.)%!"#$%34")(2,.ED(#"#%!"#$%$..'#% :"/,#$2(-.)%.G%&(2-4,1()$#% &2"1(2"%!"#$%3"##,.)% 784H()/"%.G%4.)+/92(-.)% &(2-4,1()$%;.2A',#$% I(J%$"#-)/%#"##,.)% 78"49$"%!"#$%3"##,.)% &'(K.2=%#"##,.)%!"#$%I./#%L.),$.2,)/% :"1.2$%!"#$%3"##,.)%!"#$%39==(2?%:"1.2$% Figure 7: Interoperability QMS Processes All processes and actors are described in a Quality Manual which should be known by all stakeholders in the testing process and permeate all corresponding organisations (e.g. IHE, EuroRec and, in the medium term, the implementers organisations also) and their staff. There should be websites promoting interoperability testing QMS and organisations or companies offering training and workshops on that issue. D6.4 Final Report and Roadmap Page 22 of 36

23 Table 1: Interoperability QMS Actors Interviews and questionnaires evaluated 6 at Connectathon 2011 in Pisa also indicated that the neutral persons validating the tests at Connectathon (the monitors ) expect to benefit from such a Quality Management System. Implementing organisations like IHE and EuroRec need to identify measurable quality indicators which will permit them to assess the quality of their QMS and improve relevant quality processes whenever necessary. 4.2 Test Tools Test tools are important for making large parts of a Quality Management System operational and feasible. Therefore, a tool ecosystem 7 should be established in the future that fosters active tool development and maintenance. Against this background HITCH envisages an ecosystem with the characteristics described below. First of all, an international agreement on (a set of) common standards, respectively profiles, should build a common ground for tool development, thus concentrating the efforts of the developer community. All standards developed (if necessary) are always accompanied by the development of tools needed to test their implementation in systems. This can be achieved by expressly, politically and monetarily fostering tool development within large-scale projects such as the epsos 8 project. These tools are used by the system developers as well as in testing events that take place on a regular basis. Most probably, the tools will be used in certification or labelling initiatives and programs (see section 0) and support functional as well as interoperability (including conformance) testing approaches. It is important that profiles and related tests are built on realistic use cases. The test tool development process will implement the interoperability QMS, in particular for the activities of test plan definition, test design, test tool development and selection, test case, test session execution, test session reporting and all required validation activities. 6 HITCH Deliverable HITCH Deliverable D6.4 Final Report and Roadmap Page 23 of 36

24 Tools support and reference interoperability, and functional criteria are linked to the corresponding standards. For each test to be executed, test partners (in the case of interoperability testing) can be assigned manually or by the test management tool itself, along with the necessary tool chain (e.g. validators, sniffers or simulators for missing test partners). Based on the tests run, the test management tool will enable evaluation of a system s test summary, for example, in order to find out whether it fulfils the requirements of the underlying certification program. All performed tests, partners, test results and tools used should be available within that test management tool. What is needed to address the interoperability challenge in the ehealth sector? o International agreement on common standards or profiling of standards (crossreferences/mappings between standards in the least); o Close cooperation between political mandates, SDOs, testing/auditing organisations and implementers; o Standard development systematically accompanied by test tools, already available or developed specially; o Regular testing events such as the IHE Connectathon; o Quality labelling/certification programmes that include testing of ehealth systems; o Alignment of functional auditing and interoperability testing, based on realistic and jointly agreed use cases; o An ecosystem that supports the continued existence of testing tools. Test Plan Definition o Creation and organisation of test criteria (functional statements and interoperability criteria); o Links between criteria and the underlying standards; o Composition of test plans based on those criteria. Test Design o Test design tools that combine functional requirements and interoperability criteria into a test case that can be assigned to a number of test partners and test tools (simulators, validators); o Tools aware of the concepts to be tested (model-aware) in order to guide the test designer. Test Tool Development and Selection o Develop those test tools still missing today (gap analysis); o Improve existing tools where needed. Test Case and Test Tool Validation o Make sure test software works correctly and tests comprehensively; document in validation report; o Maintain bug tracking tools for bugs in test cases and test tools. Test Session Execution o Provide training material for auditors and testers; o Integrate test tools into the test management tool. Test Session Reporting o Evaluation of the system under test: features tested, failed tests, list of nonconformities, list of third party systems used for testing; o Evaluation of the test session: number of tests performed / verified / failed / not verified; o Evaluation of the monitor work: was staffing sufficient? o Evaluation of the testing quality:! Satisfaction questionnaire for participants and monitors;! Report to the QA management in order to identify weaknesses and improvements for the next cycle. D6.4 Final Report and Roadmap Page 24 of 36

25 During and after testing, correct and sufficient tool functionality has to be checked and reported so that tools can be enhanced where needed. This also requires bug-tracking facilities. The use of free Open Source software in tool development is seen as the optimal approach, at least from the user s perspective, concentrating developer s efforts on common projects. Another major benefit is that such tools can be freely used by the testing organisations (such as companies and certifiers), as well as be checked by them in terms of security problems and functionality. However, it is understood that other business models may be applied such as dual licensing (e.g. free Open Source for not-for-profit organisations and cost-based licence for commercial applications) or offering commercial support for free Open Source tools. It is clear that many test tools from the different areas closely interact with each other, for example, test tools such as sniffers and validators produce results that are then fed into test reporting tools which consolidate them into a unified summary. The test management tool in particular (as the dashboard for testing) should and must interact with many other tools. Therefore, tools should support a common description language for input and output data. The same applies to the tools order and execution details. This is where ETSI s TTCN-3 family of standards 9 and the corresponding TTCN-3 language is seen as one example of a possible solution in the future. Finally, in any phase of the testing process, tools should support feedback regarding the auditors validating the tests, the tools used, the profiles tested, and so on. At the same time all persons participating in the testing processes must be offered training and documentation before testing. 4.3 Certification and Labelling Process The purpose is to define a labelling or certification strategy suitable to the ehealth domain. Three main scenarios already used in different domains were investigated and discussed and these represent alternatives: LabelCert of products (Third party LabelCert): The testing process is performed by a conformity assessment body (accredited third party) or a third party according to the specifications and requirements within the scope of the certification. Self-assessment with external audit: The testing process is performed by the supplier (like self-assessment), according to the specifications and requirements (that represent the scope of the LabelCert investigation) and within an agreed interoperability quality assurance framework. The vendor is subject to an external third party audit. Self-assessment of products: The testing process is performed by the vendor according to the testing specifications and requirements (that represent the scope) and under an agreed interoperability quality assurance framework. The legal framework regarding responsibility for the LabelCert process needs to be discussed by the actors involved and is therefore not within the scope of this document. The main conclusion is that the LabelCert process in ehealth depends on The maturity of the market: the more mature the market, the less a strong certification process is needed; 9 TTCN : Standards ES to ES , ES , ES and ES , see D6.4 Final Report and Roadmap Page 25 of 36

26 A flexible process encourages innovation by guiding and promoting interoperability between systems (the suppliers will focus more on their research and development of their products than on duplicate efforts in the interoperability field); The diffusion of interoperability profiles on a large scale and within time constraints is generally better supported in the scenario of self-assessment with external audit. All of these key principles need to be related to the legal framework in order to respond to the main objective of the ehealth LabelCert process, which is patient safety and the secured exchange of shared medical data. 5 RECOMMENDATIONS Based on the assessment of the state of the art and the vision, HITCH proposes the following recommendations for the interoperability labelling and certification of Healthcare information systems: 1. Drive adoption of Interoperability standards and profiles at European level 2. Drive adoption of the HITCH QMS 3. Drive closure of key gaps in interoperability testing tools 4. Encourage collaboration between the functionality-focused and interoperabilityfocused testing/labelling initiatives 5. Preserve flexibility across the proposed LabelCert schemes 6. Establish a two-level LabelCert process (European and National/project levels) These keys elements are described in the following subsections 5.1 Drive the Adoption of Profiles at European Level Today, many profiles are promoted at European level (epsos) and by national projects. In France, ASIP-Santé references profiles from the IHE IT-Infrastructure, Laboratory and Patient Care Coordination domains within the Cadre d interopérabilité. In Austria, IT- Infrastructure, Patient Care Coordination and Pharmacy are in use. In Italy, regional projects are also implementing IT-Infrastructure profiles. We recommend following the example of IHE in order to promote the adoption of profiles at European Level. HITCH notes that recognised profiles allow for national/regional customisation and thus reduce fragmentation of IT-systems and the associated cost of development and maintenance. The benefit of recognised profiles is that they encourage the development of test tools that help customers to improve their solutions. We think it is time to recognise these profiles to allow wider usage and a harmonised market. HITCH therefore recommends supporting an end-to-end-process for profiles at European level: creating profiles (when necessary, e.g. when profiles are not available at international level), from use-case to testing independent of which entity is actually performing such a process. We recommend adopting the Process for the conception, approval and recognition of the profiles and their diffusion through testing events as well as through publishing lists of conformant implementers. D6.4 Final Report and Roadmap Page 26 of 36