D6.6 Certification Framework

Transcription

1 D6.6 Certification Framework Version number 1.1 Main author Francois Fischer Dissemination level Public Lead contractor ERTICO ITS Europe Due date Delivery date CIP Information and Communications Technologies Policy Support Programme (ICT PSP) European Commission Directorate General for Communications Networks, Content and Technology Grant agreement no.: Pilot type B

2 D6.6 Certification Framework Revision and history sheet Version history Version Date Main author Summary of changes V Francois Fischer Draft V Francois Fischer Final draft V Francois Fischer Reviewed draft Name Date Prepared Francois Fischer Reviewed Guillaume Vernet Authorised Giacomo Somma Circulation Recipient Date of submission European Commission Pilot consortium Authors (full list) Francois Fischer, ERTICO ITS Europe Hasnaa Aniss, IFSTTAR Jean-Marc Blosseville, IFSTTAR Abdel Méname Hedhli, IFSTTAR Francisco Priegue, CTAG Jose-Manuel Martinez, CTAG Alvaro Arrue, IDIADA Jörn Edlich, CETECOM Thomas Reschka, CETECOM Sebastian Müller, ETSI Project co-ordinator Giacomo Somma ERTICO ITS Europe Avenue Louise Brussels, Belgium Tel.: Fax: g.somma@mail.ertico.com II Version 2.0

3 Legal Disclaimer D6.6 Certification Framework This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community. The content of this document reflects solely the views of its authors. The European Commission is not liable for any use that may be made of the information contained therein. The Compass4D consortium members shall have no liability for damages of any kind including, without limitation, direct, special, indirect, or consequential damages that may result from the use of these materials by Compass4D Consortium III Version 2.0

4 ABBREVIATIONS... 7 DEFINITIONS... 8 REFERENCES... 9 EXECUTIVE SUMMARY INTRODUCTION COMPASS4D PROJECT OVERVIEW INTENDED AUDIENCE DOCUMENT OBJECTIVES DOCUMENT STRUCTURE ANALYSIS OF THE CERTIFICATION NEEDS INTRODUCTION REQUIREMENTS FOR COOPERATIVE SYSTEMS Interoperability requirements Performance requirements Vehicle integration requirements Functional requirements CURRENT CERTIFICATION ACTIVITIES ETSI conformance testing Interoperability testing - Plugtests Car to Car Communication Consortium The DG Mobility and Transport C-ITS platform initiative GAP ANALYSIS AND NEW NEEDS SMART CITY CERTIFICATION PROPOSAL FOR COMPASS4D PROPOSED ORGANISATIONAL SCHEME INTEROPERABILITY TESTING C-ITS STANDARDS USED BY COMPASS4D CONFORMANCE TESTING INTEROPERABILITY TESTING PERFORMANCE TESTING ANTENNA PERFORMANCES GNSS PERFORMANCE TESTING FOR VEHICLE INTEGRATION INTRODUCTION ELECTRO-MAGNETIC COMPATIBILITY Radiated Emissions Radiated Immunity Conducted Emissions /01/ Version 1.0

5 5.2.4 Conducted Immunity ELECTRIC AND ENVIRONMENTAL REQUIREMENTS: World geography and climate Type of vehicle Vehicle use conditions and operating modes Equipment life cycle Vehicle supply voltage Mounting location in the vehicle FUNCTIONAL FIELD TESTING INTRODUCTION FUNCTIONAL CONFORMANCE OPERATIONAL CONFORMANCE FUNCTIONAL APPLICATION TESTING Scenario for RHW1 - Traffic Jam / Queue ahead Scenario for RHW2 - Accident/Incident ahead Scenario for RHW3 - Road works ahead Scenario for RHW4 - Bicycle/Pedestrians crossing CONCLUSIONS List of Figures Figure 1: Example of C-ITS services displayed on mobile devices Figure 2: the 7 Pilot sites in Europe Figure 3: Illustration of ITS sub-systems in ETSI EN Figure 4: C-ITS certification process Figure 5: ITS test system for the CAM protocol Figure 6: Testing environment for interoperability testing Figure 7: Anechoic chamber test setup with test antenna ring Figure 8: turntable and rotating test antenna Figure 9: Example for UHTRP measurement Figure 10: qualified Anechoic Chamber (20x12x10m) for EMC tests Figure 11: RHW2 scenario description Figure 12: Chart of the event sequence Figure 13: RHW3 scenario description Figure 14: Chart of the event sequence /01/ Version 1.0

6 List of Tables Table 1: Version of standard applied by Compass4D Table 2: Conformance testing validation framework Table 3: Test specifications for networking and transport layer Table 4: Test specifications for facility layer Table 5: Test specifications for Security Table 6: Test specifications for Interoperability Testing Table 7: Maximum allowed pulse amplitude on supply lines Table 8: Functional status relevance for the test pulses Table 9: functional conformance test pro-forma table for VIS Table 10: functional conformance test pro-form table for RIS Table 11: Operational conformance test pro-forma table for VIS nominal conditions Table 12: Operational conformance test pro-forma table for VIS overloaded network Table 13: RHW1 indicators Table 14: RHW2 indicators Table 15: RHW3 indicators Table 16: RHW4 indicators /01/ Version 1.0

7 Abbreviations Abbreviation BER BLER BTP CAM CCH CDD CEN Definition Basic Encoding Rule Block Error Rate Basic Transport Protocol Cooperative Awareness Message Control CHannel Common Data Dictionary Comitée Européen de Normalisation (European Committee for Standardisation) Central ITS station Cooperative Intelligent Transport Systems Centre for Testing and Interoperability (a department of ETSI) Cellular Telephone Industries Association (now Wireless Association) Decentralised Event Notification Message Device Under Test Energy Efficient Intersection Service Electro-Magnetic Compatibility European Standard (obsolete European Norm) European Telecommunications Standard Institute European Union Equipment Under Test Functional Performance Status Classification Global Certification Forum global navigation satellite system Information and Communication Technologies International Standardisation Organisation Intelligent Transport Systems Implementation Under Test In Vehicle System Local Dynamic Map Line Of Sight Over The Air Personal ITS station Radio Frequency Road Hazard Warning Roadside ITS station Red Light Violation Warning Society of Automotive Engineers (now SAE International) Security Signal Phase and Timing System Under Test Technical Committee Total Isotropic Sensitivity Total Radiated Power Technical Specification CIS C-ITS CTI CTIA DENM DUT EEIS EMC EN ETSI EU EUT FPSC GCF GNSS ICT ISO ITS IUT IVS LDM LOS OTA PIS RF RHW RIS RLVW SAE SEC SPaT SUT TC TIS TRP TS TTCN-3 Test and Test Control Notation (release 3) VIS Vehicle ITS station 06/01/ Version 1.0

8 Definitions In the scope of this document the following definitions are used. Certification: A process to evaluate features, performances, competences of persons, device or systems and certify they match with given requirements. Conformance Testing: Testing the extent to which an IUT is a conforming implementation (ISO IS ) Relevance area: Geographical area, one or several road section, or a traffic direction within which the ITS stations (for instance a vehicle ITS station) is concerned by the traffic event. 06/01/ Version 1.0

9 References IEEE p: IEEE Draft Standard for Information Technology - Telecommunications and information exchange between systems - Local and metropolitan area networks - Specific requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) specifications Amendment 7: Wireless Access in Vehicular Environments. R&TTE directive: ETSI TC ITS ETSI EN ETSI EN ETSI EN The Radio and Telecommunication Terminal Equipment Directive 1999/5/EC establishes a regulatory framework for placing and putting into service radio and telecommunications terminal equipment (R&TTE) on the free market. (see: The Intelligent Transport System Technical Committee at ETSI (see: Intelligent Transport Systems (ITS); Access layer specification for Intelligent Transport Systems operating in the 5 GHz frequency band; Access layer ITS 5 GHz ( _60/en_302663v010201p.pdf) Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Part 2: Specification of Cooperative Awareness Basic Service ( 02_60/en_ v010302p.pdf) Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Part 3: Specifications of Decentralized Environmental Notification Basic Service ( 02_60/en_ v010202p.pdf ETSI EN Intelligent Transport Systems (ITS); Vehicular Communications; Intelligent Transport Systems (ITS); Vehicular Communications; GeoNetworking; Part 4: Geographical addressing and forwarding for point-to-point and point-to-multipoint communications; Sub-part 1: Media-Independent Functionality ( 2.01_60/en_ v010201p.pdf) ETSI EN Intelligent Transport Systems (ITS); Vehicular Communications; Intelligent Transport Systems (ITS); Vehicular Communications; GeoNetworking; Part 5: Transport Protocols; Sub-part 1: Basic Transport Protocol ( 2.01_60/en_ v010201p.pdf) ETSI TS ETSI TS Intelligent Transport Systems (ITS); Security; Security header and certificate formats (draft available only not yet publmished) Intelligent Transport Systems (ITS); Users and applications requirements; Part 2: Applications and facilities layer common data dictionary ( 06/01/ Version 1.0

10 SAE J2735 CTIA test plan ISO UNECE R10 1_60/ts_ v010201p.pdf) SAE J2735 ( ): "Dedicated Short Range Communications (DSRC) Message Set Dictionary"; Signal Phase And Timing Message (SPAT); Map Data (MAP) ( CTIA Test Plan for Wireless Device Over-the-Air Performance, Version Sept 2014 ( source/default-document-library/ctia_ota_test_plan_rev pdf?sfvrsn=2) Road vehicles - Environmental conditions and electrical testing for electrical and electronic equipment - Part 1 to part 5 ( mple&published=on) Uniform provisions concerning the approval of vehicles with regard to electromagnetic compatibility ( ates/r010r5e.pdf) 06/01/ Version 1.0

11 Executive Summary The Compass4D project aimed at deploying sustainable smart cities cooperative services on seven different pilot sites across Europe. At the infancy of the Compass4D project, Interoperability was identified as a critical capability to ensure deployment of cooperative ITS solutions, which remains compatible across all different device suppliers, in order to warranty the deployment of sustainable services available to all road users with equipped vehicles. The present deliverable reports the results of the Compass4D task on certification, which intended to address interoperability issues, in order to ensure compatibility across suppliers and service sustainability at a global level. The deliverable starts by presenting the result of the analysis of C-ITS certification needs for smart cities cooperative services. The analysis led to identify the technical requirements, which are necessary to ensure successful operation of the C-ITS services, raising the particular challenge of ensuring interoperability across all different kinds of C-ITS stations involved. The technical requirements to be included in the certification also include performance requirements for antenna and GNSS location solutions, vehicle integration requirements and the functional requirement to validate the C-ITS services not only at the component level but also at the application and system level. The deliverable presents the ongoing initiatives to develop and foster the C-ITS certification. Comparing the requirements, resulting from the analysis of the certification needs, and the ongoing initiatives, provides the gap analysis. The current initiatives have unfortunately not led to the establishment of any certification body, which is however necessary to certify cooperative ITS devices and thus to proceed for instance with a transparent selection of suitable devices as part of public procurement procedures. The deliverable presents a recommended technical framework for cooperative ITS certification, including test procedures targeting interoperability, performances, vehicle integration and functional requirements. Furthermore the present deliverable proposes a suitable organisational framework, for the establishment of a cooperative ITS certification body. The Compass4D partners have followed the recommendation concerning the assessment of the interoperability requirements, being a critical and common challenge for the successful deployment in the context of the project, as well as a necessary condition for offering demonstrators at the ITS world Congress demos in Bordeaux to use the cooperative ITS infrastructure of the Bordeaux pilot site. The interoperability of Compass4D devices was mainly assessed during the ERTICO-ETSI Cooperative Mobile Services Plugtests events, which proposed conformance testing as well as interoperability testing. The deliverable described in details the testing procedures applicable to assess the conformity of cooperative ITS devices with the requirement resulting from the analysis of the certification needs. Therefore this deliverable presents a set of suitable technical assessment procedures as well as an organisational framework to establish the appropriate cooperative ITS certification organisation. The establishment of a certification organisation is currently part of discussions carried out in the DG MOVE C-ITS platform, so that to avoid any conflict of interest, the establishment of such an organisation was not considered by the Compass4D certification task. However, Compass4D partners have actually applied the interoperability technical assessment procedures to ensure interoperability and thus compatibility of devices across test sites, across vendors and thus strongly contribute to the sustainability of the infrastructure during and after the project life. 06/01/ Version 1.0

12 1 Introduction 1.1 Compass4D project overview The Compass4D project aims at achieving a successful deployment of three cooperative ITS services: Road Hazard Warning (RHW): it aims to reduce incidents by sending warning messages to drivers to raise their attention level and inform them about appropriate behaviour. Two types of road hazards can be distinguished: static and dynamic. Static road hazards have fixed spatial and temporal properties (e.g. road works), while dynamic road hazards can occur anywhere and/or at any time (e.g. queues and traffic jams). Red Light Violation Warning (RLVW): it aims to increase drivers alertness at signalized intersections to reduce the number of accidents or reduce the impact of accidents in case they still happen. Although the focus of the service is on red light violation it also addresses situations involving emergency vehicles as well as right of way rules at signalized intersections in a more general sense. Energy Efficient Intersection Service (EEIS): it aims to reduce energy use and vehicle emissions at signalized intersections. The major advantage of a cooperative EEIS using infrastructure-to-vehicle communication is the availability of signal phase and timing information (SPaT) in the vehicle. Presenting this information to drivers enables them to anticipate the current and upcoming traffic light state Cooperative ITS services allow the deployment of applications into vehicles, equipped to receive information sent by other vehicles or by the road equipment through road side station or traffic management centres. The received information enables displaying warning to the drivers, on vehicle HMI or on mobile devices, according to the driving context (position, heading, speed ). The information provided to the driver is intended to increase driving safety and efficiency (see Figure 1 below). Emergency Vehicle Alert Green Light Optimal Speed Advisory Stop/Start Idling Real-time Road Events Figure 1: Example of C-ITS services displayed on mobile devices Two kind of radio access are used in the Compass4D project to achieve the communication of C-ITS messages: ITS-G5 radio access (IEE p) with ETSI TC ITS communication standards Internet connectivity over regular cellular networks (2G/3G/4G). The deployment is carried out in seven cities across EU: Bordeaux, Copenhagen, Helmond, 06/01/ Version 1.0

13 Newcastle, Thessaloniki, Verona and Vigo. Figure 2: the 7 Pilot sites in Europe Beyond deploying interoperable cooperative ITS systems and the three above services for the cities, the project intends to demonstrate the positive cost-benefit of C-ITS and thus become a reference model for other cities. Interoperability between the different types of components will be ensured by using global and harmonised standards. The Compass4D service operation will continue after-project life, as project partners have signed a memorandum of understanding to continue the cooperation beyond the project completion with the aim to move from pilot to large-scale deployment for a self-sustained market. The project involves public authorities, road operators, vehicle fleets and all road transport stakeholders to navigate their way to the sustainable deployment of cooperative services. Compass4D targets global harmonisation of services and is working closely with USA and Japan. 1.2 Intended audience This document addresses methodology, solution and needs for technical performance of cooperative ITS systems and services. Therefore reading this document requires good knowledge about cooperative ITS communication technologies. This document is mostly intended for engineers to understand the approach used during the Compass4D pilot project to address interoperability issues and ensure a seamless deployment of interoperable C-ITS systems across all pilot sites. 06/01/ Version 1.0

14 1.3 Document objectives The objectives of the deliverable is to report on Compass4D activities on analysis and definition of certification needs for cooperative ITS in the context of smart cities. The cooperative ITS certification aims at deploying interoperable C-ITS solutions in all cities and providing the required level of functional and operational performances. The document will therefore report on how to assess the level of interoperability and performances of C-ITS devices. 1.4 Document structure Chapter 2 provides an analysis of the certification needs. This analysis first presents the different technical requirements to be considered for certifying cooperative ITS devices. Then the chapter reports about the current initiatives, relating to cooperative ITS certification. A gap analysis shows the gap between the existing initiatives and the identified requirements. Then a proposal for the context of Compass4D is presented with a proposal for the future organisation of a certification body. Chapter 3 presents the interoperability testing procedures, which have been applied to assess the interoperability as well as the conformity to ETSI TC ITS standards, of the Compass4D ITS devices. Chapter 4 presents suitable performance testing procedures, evaluated by test laboratories, to be applied to cooperative ITS devices. Chapter 5 presents existing vehicle integration testing procedures, based on UNECE regulation 10, which can be applied by existing test houses to cooperative ITS devices. Chapter 6 presents a proposal for functional test for cooperative ITS systems. Chapter 7 presents the conclusions. 06/01/ Version 1.0

15 2 Analysis of the certification needs 2.1 Introduction Before reaching the market, products are expected to be compliant to technical standards applying for the corresponding technologies, as well as to meet a minimum level of performance. Therefore a straightforward process is necessary to certify that these devices are meeting those requirements. Actually two main kind of processes exists to certify products: Mandatory schemes defined as part of EU or regional regulations Voluntary schemes defined by the Industry, to ensure that products reaching the market are safe, or compliant to standards, or provide minimum level of performance for instance. Concerning the mandatory schemes, the applicable EU procedure is known as the type approval. In particular, the motor vehicle type approval is relevant for the automotive domain. The motor vehicle type approval is mandatory for vehicle to reach the market, however the current motor vehicle type approval scheme does not address cooperative ITS components. Concerning the radio access (3G/LTE/ITS-G5), the R&TTE (1999/5/EC) radio equipment directives exist. Cooperative systems are necessary to provide the cooperative ITS services and are involving communicating devices, named ITS stations, implemented in different types of road transport components: Vehicles (Vehicle ITS stations or VIS) Roadside units (Roadside ITS stations or RIS) Traffic management centre (Central ITS station or CIS) Persons (Personal ITS station or PIS) According to the ETSI standards EN Intelligent Transport Systems communication architecture, the ITS stations are also part of the ITS sub-systems (see Figure 3 below). However concerning the device point of view, we rather use the ITS station term. 06/01/ Version 1.0

16 Figure 3: Illustration of ITS sub-systems in ETSI EN For Compass4D the critical challenge is to ensure seamless communication among the different ITS stations and thus to ensure successful deployment of vehicle to vehicle and vehicle to infrastructure cooperative services. In absence of suitable type approval or any other industry certification scheme at the start of the Compass4D pilot, the Compass4D description of work included a task on certification, with the main goal to address the interoperability challenge described above. Beyond ensuring interoperability between vehicles and infrastructure devices at the different pilot sites, Compass4D final demonstration added complementary challenges with regards to interoperability: Allowing non-compass4d partners to use the Compass4D infrastructure of Bordeaux pilot site for the sake of the demonstrations. Enabling Compass4D vehicles to receive and properly decode CAM (Cooperative Awareness Messages) from non-compass4d vehicles, and vice-versa. The deployment of cooperative ITS communication solutions on the infrastructure, in realistic conditions, is expected to follow public procurement procedures. As part of these procedures, buyers shall ensure that the public authority is not locked in with one particular vendors and furthermore that all public road users, when driving with an equipped vehicle, are able to use the cooperative ITS services. To match these requirements, a certification framework, allowing certifying devices which are conform to the corresponding technical requirements, is necessary. Therefore the certification task is a critical step in the Compass4D description of work to ensure the successful deployment on the pilot sites as well as the final demonstration at the ITS world congress in Bordeaux. 2.2 Requirements for cooperative systems Interoperability requirements Interoperability can be simply defined as the capability for distributed components of a system to communicate successfully and to exchange consistent data. Intelligent Transport 06/01/ Version 1.0

17 Systems are using an Open System Interconnection concept, which divides the communication feature into layers and allocate each layer a protocol. A protocol described the messages and their corresponding communication procedures between the different entities of a communicating systems, i.e. the ITS stations for cooperative ITS. The protocols are specified in Technical Specifications (TS) or in Standards (EN) drafted and published by the relevant standardisation entities, namely CEN or ETSI at EU level and ISO or SAE at international level. Concerning the cooperative ITS standards, the European Commission has launched the standardisation Mandate M/453 in 2011, to CEN CENELEC and ETSI in the field of information and communication technologies to support the interoperability of co-operative systems for Intelligent Transport in the European community. The Compass4D partners have strictly followed the standards resulting from this mandate, thus ensuring interoperability of Compass4D ITS stations with the current and future Cooperative ITS devices complying with the M/453 standards requirements. However, when pilot sites decided to use cooperative ITS component from several vendors, as usual in public procurement procedures, interoperability issues and in particular compatibility issues occurred. The analysis of the issues led to the conclusion that vendors had followed the standards, but their interpretation of the requirement differed. This shows the importance of ensuring that standards are implemented correctly but also to verify the interoperability of different product from different vendors with interoperability tests involving all devices. The interoperability requirement includes the security procedures, which are parts of the set of cooperative ITS standards Performance requirements Minimum levels of performances are necessary for deploying services, in particular relating to: Antenna performance GNSS positioning performances Antenna performance requirements are provided in many standards but also dedicated publication. However a CTIA test plan (Test Plan for Wireless Device Over-the-Air Performance) exists for the antenna performance certification, referring to the relevant standards and publications. For GNSS performance, works are currently carried out by CEN and ETSI to provide suitable performance requirement for the GNSS performances for transport application, which were however not available during the Compass4D project framework Vehicle integration requirements Implementing electronic equipment in a vehicle requires complying with UNECE regulations R10. These regulations are focusing on the two following issues: Electro-magnetic compatibility Electric and environmental requirements Functional requirements Beyond capabilities, relating to interoperability and performances, cooperative ITS devices needs also to provide suitable functional behaviours. Therefore functional requirement are necessary to be fulfilled by all devices forming the cooperative systems. Functional requirements need to focus on the procedures beyond the communication and 06/01/ Version 1.0

18 exchange of data. For instance functional requirements shall cover the performance at the application, level, ensuring the warning received by ITS station are leading to display appropriate and timely warning to the driver. Therefore ensuring the reliability of the C-ITS application, concerns the whole system, in extensor the combination of the different C-ITS devices. Functional requirements would therefore assess appropriate performances of the C-ITS services, at the application and system level, rather than at the device level. 2.3 Current certification activities ETSI conformance testing Since 2009 ETSI has carried out activities to define test specifications covering the protocol requirements from the ETSI TC ITS (Technical Committee Intelligent Transport Systems) set of standards. Consequently test procedures and the corresponding test systems exist to carry out conformance testing to assess interoperability. Thus verifying cooperative ITS station is possible by using existing test solutions based on the ETSI TC ITS test specification, but also while participating to the Plugtests event (see below) and carrying conformance test session Interoperability testing - Plugtests Since 2011, ERTICO and ETSI has joined efforts with many stakeholders to organise regulars Interoperability event for cooperative ITS devices, based on the ETSI TC ITS set of standards. These series of event is named the ETSI Cooperative Mobile Systems (CMS) Plugtests events. The interoperability events enable cooperative ITS implementation vendors to carry out 2 kind of test: Conformance test based on the test specification published by ETSI TC ITS, enabling to test ITS implementation in front of simulators executing test scripts Interoperability test during test sessions pairing 2 devices vendors checking communication procedures between 2 ITS stations Car to Car Communication Consortium The Car to Car Communication Consortium is studying the provision of certification services in order to carry out compliancy assessment of cooperative ITS systems. The testing requirements and procedures are likely to be according the chapter Requirements for cooperative systems, above. However the Car to Car Communication Consortium has announced in October 2015 postponing the initial deployment of cooperative vehicle from 2015 to 2019, so that their certification procedures are still not available The DG Mobility and Transport C-ITS platform initiative The DG MOVE C-ITS platform defines itself as following: The platform will address the main barriers and enablers identified for the deployment of C-ITS in the EU, in relation to the services likely to be introduced in the first stage (Day 1 applications) in view to provide policy recommendations to the European Commission for the development of a Communication on the Deployment of C-ITS in the EU by the end of As part of its work programme, the C-ITS platform intiative has a work package about certification. This work package (WP5) has carried out a study about Cooperative ITS. It has delivered a report at the end of 2015, recommending in particular to pursue the compliance assessment process and to establish a governing body as well as a compliance assessment body to administer the compliance assessment process. 06/01/ Version 1.0

19 2.4 Gap analysis and new needs Currently testing procedures exist to carry out conformance and interoperability testing. Performance requirement are also defined as well as the corresponding test procedures. These testing procedures aim at ensuring proper communication and exchange of consistent data. However at the functional level, new test procedures need to be defined, in order to complete the existing testing scheme and to cover the requirement mentioned in the chapter The analysis also showed that no dedicated body exist to take care of the administrative procedures, necessary to deliver and register certificate of conformity, resulting from the certification procedures. 2.5 Smart city certification proposal for Compass4D In the context of the Compass4D certification needs it was decided to focused the effort on the Interoperability challenges, which are critical to ensure the deployment of interoperable ITS devices on the different pilot sites. This target is ensuring successful deployment for the sake of the project objective, as well as the deployment of sustainable cooperative ITS services and components for the after-project life. The establishment of a certification body for cooperative ITS devices, is not feasible during the Compass4D framework and would furthermore conflict with the DG MOVE C-ITS platform initiative. However the most important challenge for the Compass4D partners, providing and installing C-ITS devices, is to ensure the conformity with the standards in order to warranty interoperability with other devices. During the Compass4D project time frame, the main goal of the certification task was therefore to ensure that all devices used at the different pilot sites are interoperable. Interoperability of devices across test site was also considered as important to fit real conditions, when drivers expect using C-ITS services while driving across Europe. For the sake of interoperability, applying the ETSI TC ITS test specification was assumed as essential. To complete the assessment for interoperability, applying for the CMS Plugtests event was assumed as the best way to carry out conformance and interoperability testing. Conformance tests provide assessment of the conformity of Cooperative ITS devices with communication requirements as specified in the relevant cooperative ITS standards. Applying to interoperability test session provide confidence on the compatibility of Compass4D vendor devices with other vendors participating in the CMS Plugtests events. Concerning the other certification procedures, test houses have been involved to study and reports about: The description of test procedures to assess the conformity of cooperative ITS devices with performance requirements The description of test procedures to assess the conformity of cooperative ITS devices with vehicle integration requirements The suitability of functional test procedures to be carried out on test racks or on open road This document presents, in the following chapters, the recommended testing procedures to be applied for a future cooperative ITS certification framework: Chapter 3: Interoperability testing Chapter 4: Performance testing Chapter 5: Testing for vehicle integration 06/01/ Version 1.0

20 Chapter 6: Functional field testing In the following chapter, a suitable organisational framework is presented, to serve as template to establish the proper cooperative ITS certification body. The combination of the technical test procedures described in the chapters 3 to 6 and the organisational framework presented in the chapter 2.6 below, form the recommended certification framework for the cooperative ITS systems for smart cities. 2.6 Proposed organisational scheme For the future, it is recommended to use a voluntary certification scheme for cooperative ITS, driven by the relevant industry players and road operators. The certification framework needs to be coordinated by an independent organisation, representing the relevant stakeholders. A C-ITS manufacturer is a corporate entity producing devices providing ITS stations features. This kind of device is then expected to be embedded in vehicles, roadside unit or central (in traffic management centre for instance), with additional software to provide C-ITS services to the road users. The following C-ITS certification scheme is based on the existing and well-established mobile terminal certification scheme from the Global Certification Forum (GCF) [ The C-ITS certification proposal is illustrated in Figure 4 and is described below. Figure 4: C-ITS certification process The Certification Body (for instance ERTICO as an independent stakeholder) is the 06/01/ Version 1.0

21 centralized entity of this proposal. Its responsibilities contain: Definition of certification criteria (based on existing standards: ETSI, SAE, etc.) handling of certification requests of C-ITS manufacturers definition of test scope for the certification (based on the device flavour and functionality) submit certificate after successful C-ITS certification maintenance of certified devices authorization of ISO17025 accredited test labs (e.g. CETECOM and other independent test labs) o based on frequent repetition of the accreditation in strict accordance on not yet defined certain criteria o nomination of qualified lab auditors maintenance of a database, which lists validated test cases and test systems The Certification Body should be accredited according to the following standard: EN ISO/IEC 17065: Conformity assessment Requirements for bodies certifying products, processes and services The ISO accredited Test Lab is responsible for: execution of test cases according to the C-ITS certification criteria testing will be performed: o by qualified persons o only on validated test systems o in a shielded lab environment validation of test cases on selected test systems creating test reports and submission to the Certification Body The Test Lab should be accredited according to the following standard: EN ISO/IEC 17025: General requirements for the competence of testing and calibration laboratories 06/01/ Version 1.0

22 3 Interoperability testing 3.1 C-ITS standards used by Compass4D Compass4D partners have agreed on the standards to be used during the deployment, based on the EC M/453 mandate on cooperative ITS standardisation. The goal was to follow the same set of standards, but also to ensure sustainability with regards to the future version of the cooperative ITS standards. Compass4D partners have therefore chosen to apply not only a recent version of the standards but also the almost stable one and the versions to be used in the latest Plugtests event planned in March This clever choice has allowed ensuring sustainability and compatibility with other cooperative ITS implementation on the market. Table 1: Version of standard applied by Compass4D Protocol Standard Version Comment Access ETSI EN PU - V /07/2013 CEN/ETSI/ISO common ITS architecture CAM ETSI EN PU - V /11/2014 Cooperative awareness here I am messages DENM ETSI EN PU - V1.2.2 Event notifications 28/11/2014 GeoNetworking ETSI EN PU - V1.2.1 Media independent Functionality (MIF) /07/2014 BTP ETSI EN PU - V1.2.1 Basic Transport Protocol /08/2014 SEC header ETSI TS Draft - V2.1.1 Based on IEEE security standards 08/06/2013 CDD ETSI TS PU - V1.2.1 Common data dictionary 16/09/2014 SPAT/MAP SAE J2735 Based on 03/2014 Signal & phase for traffic light information PU: published See references to the above standard in the Chapter References Note: even if the security standards from ETSI are mentioned in the above table, security features were not much used in Compass4D, due to the lack of maturity of necessary security devices during the Compass4D project time frame. These standards also match the version of standards used during the latest Cooperative Mobile System Plugtests event. 3.2 Conformance testing Conformance testing is the process for testing that an implementation is compliant with a protocol standard; it is realized by test systems simulating the protocol with test scripts executed against the implementation under test (IUT). The conformance test method applied by the ETSI test specifications is defined by ISO and supports a wide range of approaches for testing including the TTCN-3 test language. With this method an ITS test system emulates a peer IUT of the same layer/the same entity, thus providing a situation of communication which is equivalent to real operation between real ITS devices. The ITS test system will simulate valid and invalid protocol behaviour, and will analyse the reaction of the IUT. The test verdict, e.g. pass or fail, will depend on the result of this analysis. For instance as shown in the Figure 5 below, to test the CAM protocol, the ITS test system 06/01/ Version 1.0

23 will emulate the CAM functionality, i.e. it will use the BTP and GeoNetworking protocol in the networking & transport layer and the ITS-G5 access technology in the access layer. Figure 5: ITS test system for the CAM protocol The tables below list all conformance test specifications available for the ETSI TC ITS protocol layers. Note: to download the standards in the tables below, simply execute a web search on the standard number in the left column. Table 2: Conformance testing validation framework Standard number TR V1.3.1 Title Intelligent Transport Systems (ITS); Architecture of conformance validation framework Table 3: Test specifications for networking and transport layer Standard number TS V1.3.1 TS V1.3.1 TS V1.3.1 Title Intelligent Transport Systems (ITS); Testing; - Conformance test specifications for GeoNetworking ITS-G5; Part 1: Test requirements and Protocol Implementation Conformance Statement (PICS) pro forma Intelligent Transport Systems (ITS); Testing; - Conformance test specifications for Geonetworking ITS-G5; Part 2: Test Suite Structure and Test Purposes (TSS&TP) Intelligent Transport Systems (ITS); Testing; - Conformance test specifications for Geonetworking ITS-G5; Part 3: Abstract Test Suite (ATS) and Protocol Implementation extra Information for Testing (PIXIT) 06/01/ Version 1.0

24 Standard number TS V1.1.1 TS V1.1.1 TS V1.1.1 Title Intelligent Transport Systems (ITS); Testing; - Conformance test specifications for GeoNetworking Basic Transport Protocol (BTP); Part 1: Test requirements and Protocol Implementation Conformance Statement (PICS) pro forma Intelligent Transport Systems (ITS); Testing; - Conformance test specifications for GeoNetworking Basic Transport Protocol (BTP); Part 2: Test Suite Structure and Test Purposes (TSS&TP) Intelligent Transport Systems (ITS); Testing; - Conformance test specifications for GeoNetworking Basic Transport Protocol (BTP); Part 3: Abstract Test Suite (ATS) and Protocol Implementation extra Information for Testing (PIXIT) Table 4: Test specifications for facility layer Standard number TS V1.3.1 TS V1.3.1 TS V1.3.1 TS V1.4.1 TS V1.4.1 TS V1.4.1 TS V1.1.1 TS V1.1.1 Title Intelligent Transport System (ITS); Testing; Conformance test specification specifications for Cooperative Awareness Basic Service (CA); - Part 1: Test requirements and Protocol Implementation Conformance Statement (PICS) pro forma Intelligent Transport System (ITS); Testing; Conformance test specification specifications for Cooperative Awareness Basic Service (CA); - Part 2: Test Suite Structure and Test Purposes (TSS&TP); Intelligent Transport System (ITS); Testing; Conformance test specification specifications for Cooperative Awareness Basic Service (CA); - Part 3: Abstract Test Suite (ATS) and Protocol Implementation extra Information for Testing (PIXIT) Intelligent Transport System (ITS); Testing; Conformance test specification specifications for Decentralized Environmental Notification Basic Service (DEN); - Part 1: Test requirements and Protocol Implementation Conformance Statement (PICS) pro forma Intelligent Transport System (ITS); Testing; Conformance test specification specifications for Decentralized Environmental Notification Basic Service (DEN); - Part 2: Test Suite Structure and Test Purposes (TSS&TP); Intelligent Transport System (ITS); Testing; Conformance test specification specifications for Decentralized Environmental Notification Basic Service (DEN); - Part 3: Abstract Test Suite (ATS) and Protocol Implementation extra Information for Testing (PIXIT) Intelligent Transport Systems (ITS); Testing; Conformance test specification for Signal Phase And Timing (SPAT) and Map (MAP); Part 1: Test requirements and Protocol Implementation Conformance Statement (PICS) pro forma Intelligent Transport Systems (ITS); Testing; Conformance test specification for Signal Phase And Timing (SPAT) and Map (MAP); Part 2: Test Suite Structure and Test Purposes (TSS&TP) 06/01/ Version 1.0

25 Standard number TS V1.1.1 Title Intelligent Transport Systems (ITS); Testing; Conformance test specification for Signal Phase And Timing (SPAT) and Map (MAP); Part 3: Abstract Test Suite (ATS) and Protocol Implementation extra Information for Testing (PIXIT) Table 5: Test specifications for Security Standard number TS V1.2.1 TS V1.2.1 TS V1.2.1 Title Intelligent Transport Systems (ITS); Testing; Conformance test specification for ITS Security; Part 1: Test requirements and Protocol Implementation Conformance Statement (PICS) pro forma Intelligent Transport Systems (ITS); Testing; Conformance test specification for ITS Security; Part 2: Test Suite Structure and Test Purposes (TSS&TP) Intelligent Transport Systems (ITS); Testing; Conformance test specification for ITS Security; Part 3: Abstract Test Suite (ATS) and Protocol Implementation extra Information for Testing (PIXIT) 3.3 Interoperability testing Interoperability testing is the process for testing that devices can inter-operate. It is realized by connecting devices from different vendors and operating them, either manually or automatically, according to scenarios based on a protocol standard. Interoperability testing could be applied either as lab tests or as field tests. However interoperability testing requires a test operator to interact with the test environment and the equipment under test, which is not easy in a vehicle. Therefore interoperability testing should rather be applied in-house, unless specific driving conditions are necessary as part of the test objectives. The interoperability test currently proposed during the Plugtests events are suitable for lab testing only. Figure 6 below provides an overview of the basic concepts. 06/01/ Version 1.0

26 Note: Figure 6: Testing environment for interoperability testing Test Drivers in the above figure shall be understood as software drivers. The interoperability testing can be used as part of the certification test procedures. Furthermore, testing interoperability is part of regular testing workshops, where organisation developing ITS implementations can join to test the interoperability of their implementation. This kind of event is very well deployed in different ICT communities. The ETSI Centre for Testing and Interoperability (CTI) is used to organise interoperability testing workshop, under the brand name of Plugtests. Compass4D ITS station vendors have cooperated to the organisation and participated in all Plugtests events, held while the duration of the Compass4D project. At the last C-ITS interoperability CMS Plugtests, in March 2015, the following interoperability testing guidelines were drafted for managing the interoperability and Radio Frequency (RF) measurement test sessions: CAM DENM Target Geo Networking Security RF measurements Note: Table 6: Test specifications for Interoperability Testing Document title Intelligent Transport Systems (ITS); Testing; Interoperability test specification for Cooperative Awareness Basic Service (CA) Intelligent Transport Systems (ITS); Testing; Interoperability test specification for Decentralized Environmental Notification Basic Service (DEN) Intelligent Transport Systems (ITS); Testing; Interoperability test specification for Geonetworking ITS-G5 Intelligent Transport Systems (ITS); Testing; Interoperability test specification for ITS Security Intelligent Transport Systems (ITS); Testing; RF Measurements The above test specifications are currently not published and have therefore still no references. 06/01/ Version 1.0

27 4 Performance testing 4.1 Antenna performances The antenna performance tests, presented in this chapter, are based on the following specifications: CTIA Test Plan for Wireless Device Over-the-Air Performance, Version Sept The CTIA standard Test Plan for Wireless Device Over the- Air Performance specifies that the TRP measurement of the Equipment Under Test (EUT) is accomplished by sampling the radiated transmit power of the DUT at various locations surrounding the device. A three dimensional characterization of the transmit performance of the EUT (including the antenna) is executed by summing up the data from the spatially distributed measurements. Data points are taken every 15 degrees in the elevation (θ) and azimuth (φ) orientation. The TIS (Total Isotropic Sensitivity) representing the receiver performance of the Equipment Under Test (EUT), is measured utilizing Bit Error Rate (BER), Block Error Rate (BLER), or other error criteria. The test specification uses the appropriate error criteria to evaluate effective radiated receiver sensitivity at each spatial measurement location. Other than for transmit power measurements the data points are taken every 30 degrees in the in the elevation (θ) and azimuth (φ) orientation. It is recommended to make the antenna measurement of the IVS module in the overall surrounding environment and not as an isolated component because the car body plays a very important role in the measurement. The antenna performance with and without the vehicle body is different. Hence, the results of an isolated antenna measurement only will not reflect real operation conditions. Different operation modes of the car may be considered (Motor on/off, car entertainment system on/off, air condition on/off, etc.) as well. This is important to assess de-dense (selfinterference) effects, as an extension of the basic antenna measurements. The relevant test setup for the Antenna performance assessment is described below: Anechoic test chamber: RF shielded room that absorbs electromagnetic waves on ceiling and walls, but not on the floor. Turntable for the car Test and communication antenna: rotating test antenna or antenna ring with 15 spacing OTA (Over The Air) system software with measurement and reporting capabilities Radio communication tester to enable active testing with DUT performing a call. RF switch matrix 06/01/ Version 1.0

28 Figure 7: Anechoic chamber test setup with test antenna ring Figure 8: turntable and rotating test antenna The test procedure describes that the car needs to be rotated in its axis from 0 to 360 and the test antenna has to rotate from 0 to 90 in 15 (TRP) or 30 (TIS) steps. 06/01/ Version 1.0

29 The measurements will be performed in a semi anechoic chamber and the car will be positioned on a reflective floor and the reflections are absorbed on wall and ceiling. Alternatively, an open test environment can be applied. In such test setups upper hemisphere measurement will be sufficient. This way the (UHTRP) Upper Hemisphere Total Radiated Power and (UHIS) Upper Hemisphere Isotropic sensitivity quantities shall be calculated. The (UHTRP) measurement will look similar to the following figure: Figure 9: Example for UHTRP measurement 4.2 GNSS performance As for GNSS measurements the incidence angles of the satellites vary between the different latitudes, the route shall cover all areas of latitude where the system will be used from North of Europe to the South. The test routes shall also cover different environments like urban, interurban, regional roads, rural roads, highways and specific situations that have an effect on the GNSS accuracy or on the mobile network reception: Low mobile coverage (valleys, tunnels, forest) Interference (port, airport, high voltage cables and pylons) Low GNSS coverage (valleys, tunnels, forest, urban canyons) The different measurement sets should be collected with different satellite constellations (e.g. at different times of the day). The test region shall cover a representative part of the cities from North to South and from West to East. The route shall cover examples for all environmental conditions in Europe. The execution of the test drives and the evaluation shall be done by accredited laboratories. The same behaviour applies to measurement of antenna gain and GNSS measurement accuracy. The goal of the GNSS performance evaluation is to guarantee a minimum precision and operability of the positioning system that belongs to the IVS. A GNSS receiver needs to receive at least four signals from different satellites to determine a position. In general, one can say the more satellites and the stronger the received signals, the higher the positioning precision. The following performance requirements should be considered for the assessment of GNSS 06/01/ Version 1.0

30 receiver performance: Sensitivity: A sufficient number of satellite signals are received, all signals are weak. The goal is to ensure the reception of a minimum satellite signal level in the GNSS receiver. Nominal accuracy: A sufficient number of satellite signals are received, all signals are strong. The goal is to check the exactness of the position determined in the GNSS receiver. Dynamic range: A sufficient number of satellite signals are present but the signal level of the received signals differs a lot, i.e. there is a high dynamic range among the received signals. The goal is to test whether the weaker satellite signals are correctly received by the GNSS receiver in the presence of other strong satellite signals. Multi-path performance: A sufficient number of satellite signals are present but some satellite signals are subject to multi-path propagation, i.e. besides the direct Line-of-Sight satellite signal a weaker copy of the signal (echo or so- called multi-path signal component) arrives with some time delay at the GNSS receiver. This can occur in scenarios with large objects near the receiver, e.g. houses in an urban environment or hills in a mountainous area. Moving scenario: A sufficient number of satellite signals are received, all signals are strong. The received signal scenario represents a moving vehicle with varying speed and direction. The goal is to check whether the position is updated and sent with sufficient position precision and within an appropriate time. 06/01/ Version 1.0

31 5 Testing for vehicle integration 5.1 Introduction Electronic components integrated in a vehicle require passing through specific test procedures. As part of the UNECE (United Nation Economic Commission for Europe) regulation Nr. 10 (R10), electronic components shall fulfil EMC (Electro-Magnetical Compatibility) requirements. These tests are mandatory for all automotive components, to be embedded in a vehicle. Additionally, optional electric and environmental test are recommended. These test addressing electric, climatic, mechanical and chemical requirements should be executed in order to ensure the robustness of the electronics components. The base standard for these tests is the ISO Road vehicles - Environmental conditions and electrical testing for electrical and electronic equipment. This standard provides guidance concerning environmental conditions commonly encountered by electrical and electronic systems installed in automobiles. 5.2 Electro-Magnetic Compatibility The goal of Electromagnetic compatibility (EMC) tests is to assure the proper operation of different equipment operating in the same electromagnetic environment without causing interference and without suffering from other equipment electromagnetic radiation (immunity). In Electromagnetic compatibility the equipment should fulfil the Regulation No.10 (R10) in order to get the UNECE e-mark, which is mandatory for all vehicles and automotive components. Regulation 10 is defined by the four following test groups Radiated Emissions Radiated emissions are electromagnetic energy created by a device and released as electromagnetic fields that propagate through air, away from the car. The test shall be performed according Annex 7 (Narrowband emissions) and Annex 8 (Broadband emissions) of UN Regulation 10 rev.4. Typical test setup according CISPR 25 is shown in Annex X. Measured emissions shall be below the limits specified in sections (Broadband emissions) (Narrowband emissions) of UN Regulation 10 rev.4. Wanted emissions within necessary bandwidth are excluded from test. Consist on the evaluation of the radiated field from 30MHz to 1GHz; this test can be performed at 3 or 10 meters apart from the car. Radiated Emissions must be performed as Narrow Band and Wide Band. Narrow Band test contains only the electronic components and can be tested using the ignition key. Narrow Band test shall be performed using the average mode. Wide Band test must be performed with the switch on mode of the car. This way the engine and the ventilators will be active. In case of an electric vehicle which does not present a motor, it is needed a roller and the test will be performed with the vehicle at 40km/h speed. Wide Band test shall be performed using the peak or quasi-peak mode. Test setup specifications for radiated emissions: RF transmitters shall transmit during the test. At least a communication tester is required for communication simulation. 06/01/ Version 1.0

32 Ambient noise emissions shall be at least 10dB from the type approval limits, including emissions coming from auxiliary equipment (e.g. communication tester). Antennas (if external), wiring harness and other loads shall be representative of the vehicle installation Radiated Immunity Radiated Immunity concerns the ability of the device to operate acceptably when subjected to radio frequency Electromagnetic field radiated to the vehicle. Tests shall be performed according to Annex 9 of UN Regulation 10 Rev.4. Test setup according ISO There shall be no degradation of the ecall IVS safety functions during the test. Class A must be fulfilled. The loss of function of receivers (GNSS, GSM/UMTS) during the immunity test does not necessarily lead to fail criteria when the signal is within the receiver bandwidth (exclusion band). Radiated Immunity test shall analyse the performance of the vehicle with frequencies ranging from 2MHz to 2GHz. The Electromagnetic field strength which the device must bear shall be of 30V/m. Test setup specifications for radiated Immunity: A communication tester and vector signal generator can be used to simulate the communication link. Technical details shall be agreed between manufacturers and technical service. Free space antenna method defined in ISO is preferred. Other methods like BCI (ISO ) or DPI ( ) do not take into account coupling effects trough antenna ports since RF energy is coupled only into the wiring harness. TEM (ISO ) and STRIPLINE (ISO ) methods are limited due to EUT and auxiliary equipment size. Antennas (if external), wiring harness and other loads shall be representative of the vehicle installation. Figure 10: qualified Anechoic Chamber (20x12x10m) for EMC tests 06/01/ Version 1.0

33 5.2.3 Conducted Emissions Conducted emissions are electromagnetic energy created by a device and transmitted in the form of an electrical current through its power cord. The test shall be performed according to Annex 10 of UN Regulation 10 rev.4. Typical test setup according to ISO is shown in Annex X. The measured voltage shall be below the limits indicated in the Table 7 below. Table 7: Maximum allowed pulse amplitude on supply lines Polarity 12V systems 24V systems positive +75V +150V negative -100V -450V Conducted Immunity Conducted Immunity concerns the ability of the device to operate acceptably when subjected to radio frequency voltage or current on interconnecting conductors. The test shall be performed according to Annex 10 of UN Regulation 10 rev.4. Test setup according to ISO suitable for ecall IVS testing is shown in Annex FPSC (Functional Performance Status Classification) describes the operational status of a device during and after the exposure to an electromagnetic environment, five classes are defined: Class A: All functions of a device or system perform as designed during and after the exposure to a disturbance. Class B: All functions of a device or system perform as designed during exposure; however, one or more of them may go beyond the specified tolerance. All functions return automatically to within normal limits after exposure is removed. Memory functions shall remain class A. Class C: One or more functions of a device or system do not perform as designed during exposure but return automatically to normal operation after exposure is removed. Class D: One or more functions of a device or system do not perform as designed during exposure and do not perform as normal operation until exposure is removed and the device is reset by a simple user action. Class E: One or more functions of a device or system do not perform as designed during and after the exposure and cannot return to proper operation without repairing or replacing the device or system. For each pulse the functional criteria defined in Table 8 shall be fulfilled. 06/01/ Version 1.0

34 Test pulse Table 8: Functional status relevance for the test pulses Functional status for safety functions 1 Class C Class D 2a Class B Class D 2b Class C Class D 3a Class A Class D 3b Class A Class D 4 Class B Class D Functional status for non-safety functions The UNECE Regulation 10 supersedes the automotive EMC Directive 2004/104/EC. Additionally, radio transmitter equipment shall comply with essential requirements of Directive 1999/5/EC (R&TTE Directive). Test setup specifications for conducted Immunity: A communication tester and vector signal generator can be used to simulate the communication link. Technical details shall be agreed between manufacturers and technical service. Direct cable connection to the communication tester is not recommended, external protections fitted on the communication tester could influence test results. Transient pulses shall be applied to all supply lines (e.g. ACC, IGN, BAT) simultaneously and independently. IVS antennas (if external), wiring harness and other loads shall be representative of the vehicle installation. 5.3 Electric and environmental requirements: The ISO set of standards applies to electric and electronic systems/components for vehicles. It describes the potential environmental stresses and specifies tests and requirements recommended for the specific mounting location on/in the vehicle. ISO is divided in the following parts, which considers the conditions stated below. ISO (2006) General ISO (2010) Electrical Loads ISO (2007) Mechanical Loads ISO (2010) Climatic Loads ISO (2010) Chemical Loads The concept of ISO is to assist its user in systematically defining and/or applying a set of internationally accepted environmental conditions, tests and operating requirements, which are based on the anticipated actual environment in which the equipment will be operated in and exposed to during its life cycle World geography and climate Road vehicles are owned and operated in nearly all land regions of the earth. Significant variation in environmental conditions due to climatic environment, including diurnal and seasonal cycles, can therefore be expected. Consideration has been given to worldwide ranges in temperature, humidity, precipitation and atmospheric conditions including dust, pollution and altitude. 06/01/ Version 1.0

35 5.3.2 Type of vehicle Environmental conditions in an on road vehicles can depend on vehicle design attributes, such as engine type, engine size, suspension characteristics, vehicle mass, vehicle size, electrical supply voltage and so on. Consideration has been given to typical types of vehicles including commercial (heavy) trucks, passenger cars and trucks and diesel and gasoline engines Vehicle use conditions and operating modes Environmental conditions in and on the vehicle vary significantly with road quality, type of road surface, road topography, vehicle use (e.g. commuting, towing, cargo transport, etc.) and driving habit. Operating modes, such as storage, starting, driving, stopping and so on, have been considered Equipment life cycle Electrical and electronic equipment are also resistant to environmental conditions experienced during manufacture, shipping, handling, storage, vehicle assembly and vehicle maintenance and repair Vehicle supply voltage Supply voltage varies with vehicle use, operating mode, electrical distribution system design and even climatic conditions. Faults within the vehicle electrical system, such as overvoltage alternator and intermittencies in connection systems, may occur Mounting location in the vehicle In current or future car concepts, systems/components are mounted in almost any location of the car. The environmental requirements for each specific application highly depend on its mounting location. Each location in a vehicle has its distinct set of environmental loads. As an example, the range of temperatures in the engine compartment differs significantly from the range in the passenger compartment. This is also true for the vibration loads. But in this case, not only the vibration levels are different; the type of vibration load also varies. Body mount components are typically exposed to random vibrations whereas for engine mount systems/components the additional sine vibration from the engine is considered. Devices installed in doors are exposed to a high number of mechanical shocks from door slamming additionally. 06/01/ Version 1.0

36 6 Functional field testing 6.1 Introduction The test procedures, presented in the above chapters, are intended to be performed in laboratories. However performing evaluation of the cooperative ITS system performance in real driving conditions are expected to assess complementary features, being essential for the successful execution of the cooperative applications. Therefore these validation procedures are enabling assessing the performances of the complete systems, including vehicle, roadside and central ITS stations. The procedures are therefore not assessing performance of isolated devices but the performances of the whole systems, while giving the opportunity to identify which devices is responsible for faulty behaviour. The test can be performed on private tracks, equipped with cooperative ITS equipment, including a crossing with traffic lights, or on open road. These are the reasons for calling this group of tests functional field testing. Dedicated test solutions are deployed on the test tracks to collect different kind of data including: ITS messages sent by vehicles and roadside stations Vehicle measurements (speed, position ) All data are collected by a central system running scenarios in parallel to the driving vehicles enabling to evaluate performances relating to given indicators. 6.2 Functional conformance These tests allow verifying the communication capabilities from a functional point of view, meaning during real drive tests. These capabilities are necessary to deploy ITS service functions Table 9 below shows a pro-forma table to report test result about the functional conformance of Vehicle ITS station. Table 9: functional conformance test pro-forma table for VIS Functional Conformance Environment: Closed test tracks VIS Test date Scenario Identification Title Report Result VIS Scenario1 CAM generation VIS Scenario2 CAM reception VIS Scenario3 DENM generation (manual) from VIS VIS Scenario4 DENM generation (automatic) from VIS VIS Scenario5 DENM reception from VIS VIS Scenario6 DENM reception from CIS VIS Scenario7 Detection of a record in LDM data collection for test VIS Scenario8 Information sent to HMI Table 10 below shows a pro-forma table to report test result about the functional 06/01/ Version 1.0

37 conformance of Roadside ITS station. Table 10: functional conformance test pro-form table for RIS Functional Conformance Environment: Closed test tracks RIS Test date Scenario Identification Title Report Result RIS Scenario1 CAM reception and transferred to CIS RIS Scenario2 DENM reception and transferred to CIS RIS Scenario3 DENM generation by CIS 6.3 Operational conformance These tests allow verifying the performance of the cooperative ITS systems in order to allow service operations. Table 11 below shows a pro-forma table to report test result about the operational conformance of Vehicle ITS station with nominal conditions Table 11: Operational conformance test pro-forma table for VIS nominal conditions Operational Conformance Environment: Closed test tracks VIS Test date Scenario Identification Title Report Result Operational 1 Minimum range in LOS (exigency > 300m) Operational 2 Packet error rate for CAM exchange in LOS Operational 3 Measure of average and maximum age of dynamic data stored in the LDM Operational 4 Packet error rate for DENM exchange in LOS Operational 5 End-to-End Latency average and maximum between VIS Operational 6 End-to-End Latency average and maximum between VIS and CIS The operational conformance sheet for VIS in overloaded radio network is described in 06/01/ Version 1.0

38 Table 12. It is centred on performances of VIS when they are involved in a network with a large quantity of data exchanges. The maximum rate to overload the network is to have hundred VIS in the same area sending CAM at 10 Hz. 06/01/ Version 1.0

39 Table 12: Operational conformance test pro-forma table for VIS overloaded network Operational Conformance Environment: Closed test tracks VIS Test date Scenario Identification Title Report Result Overloaded 1 Packet error rate for CAM exchange in LOS at different level of data exchange Overloaded 2 Data rate in CCH at different level of data exchange Overloaded 3 End-to-End Latency average and maximum between VIS Overloaded 4 End-to-End Latency average and maximum between VIS and CIS Overloaded 5 Measure of average and maximum age of dynamic data stored in the LDM at different level of loaded network 6.4 Functional application testing This chapter presents functional test applying in the context of the execution of ITS service applications. The test scenarios are presenting test descriptions for the following Road Hazard Warning (RHW) applications: Traffic Jam / Queue ahead - RHW 1 Accident/Incident ahead - RHW2 Road works ahead - RHW3 Bicycle/Pedestrians crossing - RHW Scenario for RHW1 - Traffic Jam / Queue ahead The purpose of this application is to warn drivers about the presence of traffic Jam. In the present chapter, the traffic jam is detected at the vehicle level. The application shall request the DEN basic service to transmit DENMs if event triggering condition is met (from ETSI TS ). Event triggering conditions: 1. Detection of a traffic jam formation through some consecutive emergency or strong brakes. 2. Detection of a stationary traffic jam through the vehicle being stationary during a predefined time, surrounded by other vehicles. 3. Detection of a traffic jam ending through some consecutive accelerations followed by stable normal speeds. The Vehicle ITS-S shall distinguish a stationary condition due to a traffic jam from any other normal, e.g. stop at a traffic lights or abnormal immobilization such as breakdown or post-crash. The Vehicle ITS-S shall distinguish a stationary condition due to a traffic jam from any other normal, e.g. stop at a traffic lights or abnormal immobilization such as breakdown or postcrash. The relevance area should be as far as possible determined according to the topology and regulatory speed limit of the road in which the signalled traffic event is located. 06/01/ Version 1.0

40 The event termination conditions are: 1. At least one DENM shall be sent when detecting a traffic condition evolution. So, the terminating condition is automatically given by this rule. 2. At least one DENM shall be sent when detecting a stationary traffic. So, the terminating condition is automatically given by this rule Description of the test scenarios In all test scenarios of this use case, a minimum of 3 vehicles equipped is required, no RIS nor CIS Scenario RHW1- SC1: traffic jam triggering conditions Vehicles fulfil the triggering condition, the following vehicle receives a DENM and provides information to the driver. Scenario RHW1- SC2: traffic jam triggering conditions SC2: only one vehicle fulfils triggering condition, vehicles receive the DENM but no information is send to the driver. Scenario RHW1- SC3: traffic jam and stationary vehicle A vehicle is stationary. A DENM is sent but no information about a traffic jam is displayed to the driver. Scenario RHW1- SC4: traffic jam notification A traffic jam is signalled. The driver has to be warned if the vehicle is in the relevance area Indicators Table 13: RHW1 indicators Name Calculation method Expected value Start from the event triggering conditions SC1-I1 V2V Latency from the time when the first vehicle <300 msec sends a traffic jam notification and the DENM reception SC1-I2 V2V Latency from the time when the second vehicle sends a traffic jam notification and the display of information to the driver when entering in the relevance area. <300msec Scenario for RHW2 - Accident/Incident ahead The purpose of this application is to warn drivers about the presence of one or multiple crashed vehicles ahead. The warning allows the drivers to avoid or mitigate collisions with these crashed vehicles. This function is realized by having a crashed vehicle transmit periodically DENM in order to inform other vehicles about its status. When it is detected that the crashed vehicle and the approaching vehicles are in a potential distance for a rear-end collision, a visual and audio signal will be delivered to the driver in the approaching vehicles 06/01/ Version 1.0

41 Figure 11: RHW2 scenario description In all test scenarios of this use case, a minimum of two vehicles equipped is required. When the target vehicle is stationary due to a crash, it periodically transmits the appropriate and corresponding DENM. Event triggering conditions: The RHW application shall request the DEN basic service to transmit DENMs if at least one of the following event triggering conditions are met: 1. Vehicle driver or passenger in an emergency situation requiring stopping immediately the vehicle. 2. Vehicle problem (breakdown) requiring some technical support. 3. Vehicle in accident. 4. Stationary vehicle with hazard lights on The functional tests described in this document take only into account the event vehicle in accident. Relevance area: 1. The location of the event 2. The relevance Distance 3. Relevance Traffic Direction 4. Destination area (shape of relevance area) Description of the test scenarios Scenario RHW2 - SC1: subject vehicle within the relevance area but goes straightforward The subject vehicle is getting into the relevance area but goes straightforward; it receives a DENM message from the target vehicle. In this case no information must be displayed to the driver of the subject vehicle. 06/01/ Version 1.0

42 Scenario RHW2 - SC2: subject vehicle within the relevance area and turns left The subject vehicle is getting into the relevance area and is planning to turn left; it receives a DENM message from the target vehicle. An alert is displayed to the driver, as he will reach the crashed vehicle after his left turn. Scenario RHW2 SC3: subject vehicle is out of the relevance area The subject vehicle is outside of the relevance area and receives a DENM message from the target vehicle. In this case no information must be displayed to the driver. Scenario RHW2 SC4: subject vehicle is leaving the relevance area The subject vehicle is leaving the relevance area and receives a DENM message from the target vehicle. In this case no information must be displayed to the driver. Scenario RHW2 - SC5: Idem SC2 but check if DENM is relayed to a third vehicle A third vehicle is getting into the relevance area; it receives a DENM message relayed from the subject vehicle. An alert is displayed to the driver. Scenario RHW2 SC6: Target vehicle no longer stationary When the target vehicle is no longer stationary, all vehicles within the relevance area must end the driver alert. Figure 12: Chart of the event sequence Indicators Table 14: RHW2 indicators Name Calculation method Expected value Start from the event triggering conditions SR02-I1 V2V Latency from the time when the first vehicle <1 sec becomes stationary and the display of information to the driver when entering in the relevance area. SR02-I2 V2V Latency from the time when the first vehicle becomes stationary and the moment when the second vehicle relays the transmission of DENM? 06/01/ Version 1.0

43 SR02-I3 End to End V2V Latency via vehicle relay 1 sec Scenario for RHW3 - Road works ahead The purpose of this application is to inform the driver about roadwork areas on his path ahead. The driver will receive a visual and audio message when approaching an area with roadworks. Thus, roadwork information is first supplied by the TCC (Traffic Control Center) to the RIS. The RIS then broadcasts the information to the VIS located into the relevance area and going in the right direction. A CIS, a RIS and at least one VIS are required to test this use case. Relevance area: geographic area in which information concerning the event is identified as relevant for use or for further distribution. Relevance area is a combination of RelevanceDistance and RelevanceTraffic Directions. Figure 13: RHW3 scenario description The following Data elements are transmitted by the DENM: Geographic positions of road work area, Total number of lanes, ID of closed lanes, Reduced speed limits per remaining open lane. The relevance area is defined by a combination of the following data (see previous ): 1. The location of the event 2. The relevance Distance 3. Relevance Traffic Direction Description of the test scenarios Scenario RHW3 - SC1: VIS within the relevance area The CIS generates a road Roadwork event: it sends information related to the Road Work to RIS in a message written in DATEX II format. The RIS receiving the DATEX II message generates and broadcasts DENM messages to vehicles located in the relevance area. The VIS shows to the driver the information related to the Road Work while in the relevance area. Scenario RHW3 - SC2: VIS is out of the relevance area The VIS is located upstream of the relevance area. In this case no information must be displayed to the driver. Scenario RHW3 - SC3: VIS leaves the relevance area 06/01/ Version 1.0