D5.3.1 Motorcycle application definition design Motorcycle application definition design Deliverable n. D5.3.1 Motorcycle application definition design Sub Project SP5 Integration and tests Workpackage WP53 Motorcycle applications Task n. T5.3.1 Motorcycle application integration Authors File name Status Distribution Pasi Pyykönen, Matti Kutila Aki Lumiaho, Kari Hanski, Tommi Lumiaho Arto Kyytinen VTT Ramboll TTS DESERVE_D53.1_Motorcycle_Application_Definition_Design.doc Final (RE) Issue date 31.7.2013 Creation date 21.5.2013 Project duration start and 1 st of September, 2012 36 months
REVISION AND HISTORY CHART VER. DATE AUTHOR REASON 1.0 21.5 Pasi Pyykönen First draft of Rider Monitoring Platform Definition Design 2.0 6.6 Pasi Pyykönen Figures of platform 3.0 15.6 Pasi Pyykönen Content update 4.0 1.7 Pasi Pyykönen Update to content. 4.1 4.7 Pasi Pyykönen Minor updates. 4.2 12.7. Aki Lumiaho, Kari Hanski, Tommi Lumiaho Addition of Safety and Convenience Application Platform Definition Design 4.3 15.7 Pasi Pyykönen Update to Figures. 4.4 16.7 Pasi Pyykönen Minor updates for the review. 4.5 31.7 Pasi Pyykönen Updates from internal review. 4.6 31.31.7 Matti Kutila Overall revision and changing status to final Version 4.6, 31/07/2013 Page 2 of 16
TABLE OF CONTENTS REVISION AND HISTORY CHART...2 LIST OF FIGURES...4 EXECUTIVE SUMMARY...5 1. INTRODUCTION...6 1.1 OBJECTIVES AND SCOPE OF THE DOCUMENT...6 1.2 STRUCTURE OF THE DELIVERABLE...6 2. MOTORCYCLE PLATFORM HARDWARE...7 2.1 CONTENTS OVERVIEW...7 2.2 RIDER MONITORING ENVIRONMENT...7 2.3 SIMULATOR ENVIRONMENT...9 2.4 SAFETY AND CONVENIENCE ENVIRONMENT... 10 3. MOTORCYCLE PLATFORM SOFTWARE... 11 3.1 RIDER MONITORING ENVIRONMENT... 11 3.1.1 Rider monitoring camera... 11 3.1.2 Head and eye detection... 12 3.1.3 Visual and cognitive distraction detection... 12 3.1.4 Local database... 12 3.1.5 NMEA (GPS) Transmitter... 13 3.2 SIMULATOR ENVIRONMENT... 13 3.2.1 Head and eye detection... 13 3.2.2 Visual and cognitive distraction detection... 14 3.2.3 NMEA (GPS) Transmitter... 14 3.2.4 Simulator database... 14 3.3 SAFETY AND CONVENIENCE SOFTWARE PLATFORM... 14 3.3.1 CAN Bus Interface... 15 3.3.2 On-Board Computer... 15 3.3.3 ITS Communication Device... 16 4. CONCLUSIONS... 16 Version 4.6, 31/07/2013 Page 3 of 16
LIST OF FIGURES FIGURE 1: RAMBOLL DEMONSTRATION POWERED TWO-WHEELERS TO HOST THE HARDWARE AND SOFTWARE PLATFORM ERROR! BOOKMARK NOT DEFINED. FIGURE 2HARDWARE COMPONENTS OF RIDER MONITORING ENVIRONMENT. HARDWARE CONFIGURATION CONSIST OF ONE USB CAMERA, AUTOMOTIVE PC AND GNSS (GPS) UNIT. 8 FIGURE 3 INSTALLATION OF RIDER MONITORING HARDWARE IN MOTORCYCLE PLATFORM. RIDER MONITORING CAMERA IS POINTING TO RIDER. 8 FIGURE 4: HARDWARE COMPONENTS OF SIMULATOR ENVIRONMENT. HARDWARE CONFIGURATION CONSIST OF ONE OR MORE USB CAMERAS, AUTOMOTIVE AND SIMULATOR PC 9 FIGURE 5: AS AN EXAMPLE FROM A TRUCK COCKPIT, INSTALLATION OF DRIVER MONITORING HARDWARE IN SIMULATOR ENVIRONMENT. DRIVER MONITORING CAMERA (1) IS POINTING TO VEHICLE S DRIVER. 10 FIGURE 6: HARDWARE COMPONENTS FOR SAFETY AND CONVENIENCE PLATFORM : CAN BUS INTERFACE, ON- BOARD COMPUTER AND COMMUNICATION DEVICE 10 FIGURE 7: SOFTWARE COMPONENTS OF MOTORCYCLE APPLICATION PLATFORM. 11 FIGURE 8: CLASSIFIED GAZE DIRECTION AREAS IN SIMULATOR ENVIRONMENT FOR GAZE DIRECTION DETECTION. 12 FIGURE 9: SOFTWARE COMPONENTS OF SIMULATOR APPLICATION PLATFORM. 13 FIGURE 10: SOFTWARE COMPONENTS ARE EMBEDDED IN THE HARDWARE MODULES OF THE DEMONSTRATION PTWS 14 FIGURE 11: CAN BUS INTERFACE RECEIVES VARIOUS SIGNALS FROM PTW SUB-SYSTEMS 15 FIGURE 12 ON-BOARD COMPUTER SCHEME FOR SOFTWARE COMPONENTS 16 Version 4.6, 31/07/2013 Page 4 of 16
EXECUTIVE SUMMARY This document presents a specification for powered two-wheeler (PTW) application definition design. In this document the hardware and software specification for PTW platform are described. The hardware solution consists of rider monitoring platform (RMP) and safety and convenience application platform (SCAP). The Rider Monitoring hardware platform is designed to include On-board Computer, GPS receiver and driver monitoring cameras. The Safety and Convenience hardware platform is designed to consist of CAN Bus Interface, On-board Computer and ITS Communication Module. Together these two platforms are designed to establish the DESERVE PTW Platform. For practical reasons the current target is to separate the two platforms since the RMP is used for only one application and thus in use far less than SCAP that hosts a selection of applications that are used far more regularly. Version 4.6, 31/07/2013 Page 5 of 16
D5.3.1 Development Platform 1. INTRODUCTION 1.1 Objectives and scope of the document The objective of this document is to describe hardware and software specification for DESERVE PTW platform. This document also includes specification for TTS simulator environment which design is related to PTW platform. Difference between these platforms is static implementation of simulator environment compared to PTW moving platform. Also amount of needed driver monitoring cameras will vary between these implementations. Objective of this document is to show difference and consistency of design of these different platforms. In this document main focus is on PTW application design. Simulator platform implementation and application design is descried if implementation is differing from PTW platform. The document is intended to verify that the motorcycle pilot meets the technical requirements and is accepted for the SP5 Integration and Test. The tests will follow and are comparable with the passenger car and commercial vehicles tests executed in WP5.1 and WP5.2. This document is related to the Vehicle interior observation application needs described in D11.2. System described in this document provides driver status as application needs for vehicle interior observation applications. These applications are Driver impairment warning System, Driver visual Distraction Warning System and Occupant Detection and Classification System. Needs for D11.2 platform application are driver head and eye detection and driver viewing direction detection. 1.2 Structure of the deliverable Figure 1: RAMBOLL Demonstration powered two-wheelers to host the hardware Version 4.6, 31/07/2013 Page 6 of 16
and software platform In section 1 is presented a short introduction of this document. Section 2 describes hardware specifications of the simulator, rider monitoring platform, RMP and safety and convenience application platform, SCAP. Section 3 describes application specifications of the simulator and PTW environments. 2. MOTORCYCLE PLATFORM HARDWARE 2.1 Contents overview For RMP, i.e. motorcycle and simulator platform, hardware consists of RAMBOLL Demonstration powered two-wheeler (PTW), driver monitoring camera and automotive PC as processing unit. Implementation of hardware differs from camera layout, number of cameras and platform computation units. In motorcycle, driver monitoring hardware is used to collect data from cameras and motorcycle hardware. In simulator platform, driver monitoring hardware is also connected to simulator PCs to exchange information between these systems. For SCAP, the hardware consists of RAMBOLL Demonstration powered two-wheeler (PTW), CAN Bus Interface, ITS On-board unit and ITS Communication unit. The PTW-specific ITS On-board unit manages and operates CAN Message-based information. In order to include also PTWs without CAN Bus in the scope of the project a CAN Bus interface is needed to interpret analogue and digital signals into CAN message format and content. The ITS Communication unit is needed for the standards-based communication between the ITS mobile station and back office solutions. 2.2 Rider monitoring environment In Fig. 2 is a shown hardware component of the motorcycle platform. Hardware consists of one USB 2.0 camera providing 720p resolution still images. Camera is connected to fanless automotive PC running Window7 32bit operating system. For vehicle positioning, GPS receiver is used trough USB 2.0 connection. Automotive PC is running Microsoft SQL 2010 database to store measurements and analysis results from rider detection applications. Version 4.6, 31/07/2013 Page 7 of 16
Figure 2: Hardware components of rider monitoring environment. Hardware configuration consist of one USB camera, automotive PC and GNSS (GPS) unit. Rider monitoring camera is installed in handlebar of the motorcycle pointing towards to motorcycle driver. Automotive PC and GPS receiver are installed to the frame and top-box of the motorcycle Fig. 3. Figure 3: Installation of rider monitoring hardware in motorcycle platform. Rider monitoring camera is pointing to rider. Version 4.6, 31/07/2013 Page 8 of 16
2.3 Simulator environment In Fig. 4 is a shown hardware component of the simulator platform. Hardware consists of one or more USB 2.0 cameras providing 720p resolution still images. In the simulator environment, several cameras are needed to detect driver s face in multiple positions i.e. when turning head to see out of side window. Cameras are connected to fanless automotive PC running Window7 32bit operating system. Automotive PC is connected through Ethernet connection to the simulator PC running Windows XC 32bit. Simulator PC is running Microsoft SQL 2010 database to store measurements and analysis results from driver detection applications. Simulator PC simulates also GPS position and provides coordinates via NMAE transmitter trough Ethernet connection to the automotive PC. Figure 4: Hardware components of simulator environment. Hardware configuration consist of one or more USB cameras, automotive and simulator PC Driver monitoring cameras are installed to the dashboard of the simulator and both side mirrors as shown in Fig 5. Automotive PC is installed inside simulator vehicle. Database/simulator PC is simulated from simulator control. Version 4.6, 31/07/2013 Page 9 of 16
Figure 5: As an example from a truck cockpit, installation of driver monitoring hardware in simulator environment. Driver monitoring camera (1) is pointing to vehicle s driver. 2.4 Safety and convenience environment This hardware platform is targeted to provide a single on-board data access and data management point for ITS applications. The ITS On-Board Computer is based on Linux Computer, ARM processor, mobile flash memory, two CAN ports I/O for sensors and external connections and communication links over Bluetooth and Ethernet interfaces. ITS CAN Interface reads analogue and digital signals through integrated data ports. It is based on operational amplifiers for analog signals, pulse shapers for digital signals and I/O module between PTW and the On-Board Computer inputs (Figure 6). Figure 6: Hardware components for Safety and Convenience Platform : CAN Bus Interface, On-Board Computer and Communication device Version 4.6, 31/07/2013 Page 10 of 16
3. MOTORCYCLE PLATFORM SOFTWARE The PTW Platform software components are located in the relevant hardware platform module, namely rider monitoring, simulator and safety and convenience modules. Rider monitoring and simulator platform software consists of software components on VTT automotive PC and VTT simulator environment. In motorcycle environment, main focus is to collect data from motorcycle GPS sensor and no additional software components in motorcycle is needed. 3.1 Rider monitoring environment In Fig. 6 are shown software components of the rider monitoring application platform. Application platform is divided in three modules: rider monitoring camera, monitoring platform and motorcycle platform. Rider monitoring camera includes camera capturing software, which provides 720p time stamped still images to monitoring platform. This platform contains NMEA transmitter to provide GPS data to motoring platform. Image and GPS data is processed in monitoring platform to gaze direction coordinates and parameters of distraction. Figure 7: Software components of motorcycle application platform. 3.1.1 Rider monitoring camera Driver monitoring camera includes camera capturing software, which provides 720p time stamped still images to monitoring platform. Camera images are pre-processed and optimised of head and eye detection in camera module. Version 4.6, 31/07/2013 Page 11 of 16
3.1.2 Head and eye detection Head and eye detection module receives captured images trough USB 2.0 from the camera module. Module also listens UDP data from rider monitoring platform NMAE transmitter. If GPS data is received from NMAE transmitter, head and eye detection starts analysing image data from the driver monitoring camera. As a output, head and eye detection provides x and y pixel coordinates in image are as a gaze direction. Head and eye detection calculates also area of the gaze direction in driving environment. In Fig. 7 are shown here as an example a view from truck cockpit classification areas marked from left to right as left side window (1), left mirror (2), left windscreen (3), right windscreen (5), dashboard (4), right mirror (7), right side window (8). 1 3 5 6 2 7 4 Figure 8: Classified gaze direction areas in simulator environment for gaze direction detection. For the PTW simulator application of eye detection the gaze areas are located differently and do not contain all of the above mentioned gaze area. Also the dashboard area is divided into smaller sub-areas that will be defined during the design phase. The actual PTW application will be designed and implemented later in the project. 3.1.3 Visual and cognitive distraction detection Visual and cognitive distraction detection module (VCDD) module receives gaze detection coordinates from head and eye detection module. These coordinates is used to calculate visual and cognitive distraction as and distraction index. If drives gaze direction is not varying enough, module will decrease of distraction index for driving. 3.1.4 Local database Local database is used to store measurements from VCDD module and head and eye detection module. Database will also store timestamps of the measurements and GPS position from the NMAE transmitter. Local database is part of the monitoring platform and communicated in local host network. Version 4.6, 31/07/2013 Page 12 of 16
3.1.5 NMEA (GPS) Transmitter In motorcycle application platform, NMEA transmitter reads GPS data from the GPS receiver and provides it to monitoring platform and the head and eye detection module trough UDP socket. Monitoring platform waits GPS coordinates from UDP socket and start measurements when GPS coordinates is detected. 3.2 Simulator environment In Fig. 8 are shown software components of the simulator application platform. Application platform is divided in 3 modules: Driver monitoring cameras, monitoring platform and simulator platform. Driver monitoring cameras includes camera capturing software, which provides 720p time stamped still images to monitoring platform. Simulator platform contains NMEA transmitter to provide simulated GPS data to motoring platform. Image and GPS data is processed in monitoring platform to gaze direction coordinates and parameters of distraction. These measurements are stored to simulator database in simulator platform. Figure 9: Software components of simulator application platform. 3.2.1 Head and eye detection Head and eye detection module receives captured images trough USB 2.0 from the camera modules. Module also listens UDP data from rider monitoring platform NMAE transmitter. If GPS data is received from NMAE transmitter, head and eye detection starts analysing image data from the driver monitoring cameras. In the simulator environment, head and eye detection calculates also area of the gaze direction in driving environment. In Fig. 7 are shown here as an example a view from truck cockpit classification areas marked from left to right as left side window (1), left mirror (2), left windscreen (3), right windscreen (5), dashboard (4), right mirror (7), right side window (8). In the simulator environment multiple cameras are needed to cover all areas for reliable head and eye detection. Version 4.6, 31/07/2013 Page 13 of 16
3.2.2 Visual and cognitive distraction detection Visual and cognitive distraction detection module (VCDD) module function is same as described in chapter 3.1.3 in motorcycle application platform. 3.2.3 NMEA (GPS) Transmitter In simulator application platform, NMEA transmitter simulates GPS data from simulator and delivers it to monitoring platform and the head and eye detection module trough UDP socket. Monitoring platform waits GPS coordinates from UDP socket and start measurements when GPS coordinates is detected. Sending of the GPS coordinates starts when simulator platform needs to start monitoring with driver monitoring software. 3.2.4 Simulator database Simulator database is used to store measurements from VCDD module and head and eye detection module. Database will also store timestamps of the measurements and GPS position from the NMAE transmitter. Simulator database is installed as part of the simulator environment and can be used for playback of driving situation. 3.3 Safety and convenience software platform In total the safety and convenience software platform resides in various parts installed in PTW, Figure 10. Figure 10: Software components are embedded in the hardware modules of the demonstration PTWs Version 4.6, 31/07/2013 Page 14 of 16
For this document the most relevant software modules are parts of the CAN Bus Interface and On-Board Computer. CBI hosts the software tools to interpret digital and analog signals of the PTW sub-systems into CAN message format and content. Interpretation tools utilise operational amplifiers and pulse shapers in order to achieve CAN-compatible messaging between PTW and OBC. The OBC is connected to CBI via fieldbus. 3.3.1 CAN Bus Interface The data coming from the PTW sub-systems contains six analog signals (O 2 sensor, throttle position, manifold air pressure, intake air temperature, coolant temperature and front brake pressure), two frequency signals (RPM, speed) and four digital signals (brake light, turn signals and steering angle) (Figure 11). CBI provides these contents via CAN Bus to OBC and PTW sub-systems. The data can be then used for various safety and convenience application to initiate information, warnings, contextual information and eventually advice for suggested rider action. Figure 11: CAN Bus Interface receives various signals from PTW sub-systems 3.3.2 On-Board Computer OBC utilises information/data that it receives from two fieldbus ports, USB port and I/O ports. Linus computer powered by ARM Processor receives, handles and manages all data transport. The data is stored in a flash memory disc. All communications from PTW takes part over Bluetooth profiles and Ethernet cable where a separate and specific ITS over IPv6 device is connected. Version 4.6, 31/07/2013 Page 15 of 16
Figure 12: On-Board Computer scheme for software components 3.3.3 ITS Communication Device ITS Communication Device is targeted to handle all communications according to the established relevant CEN/ETSI standards. This brings available several communications protocols like IEEE 802.11n and 802.11p as well as cellular networks (3/3,5G) and LTE (4G) networks. 4. CONCLUSIONS This document introduced applications specific definition design of DESERVE motorcycle platform. In this document we showed differences and consistency of simulator and onboard rider monitoring application platform design as well as the safety and convenience application hardware and software platform. Main difference of the simulator and on-board rider monitoring scenarios is different sensor layout and way of store data from rider monitoring applications. On the other hand, safety and convenience application hardware and software platform collects, interprets, manages and thus enables development of various PTW rider applications for safer, economic and environmental friendly riding of powered two-wheelers. Version 4.6, 31/07/2013 Page 16 of 16