Approach of In-Vehicle Infotainment development on open source software. May22-24, Francisco Takeshi Hoshina

Transcription

1 Tizen Developer Conference Approach of In-Vehicle Infotainment development on open source software May22-24, Francisco Takeshi Hoshina 1

2 Self-introduction

3 Agenda What is IVI system? How We collaborate with Tizen IVI Introduction of Toyota source code UI Manager Vehicle Information Control (VIC) Home screen Summary Q&A 3

4 What is IVI system? Portable Device IVI Other ECU RSE Rear view camera Example In-Vehicle Infotainment (IVI) system i.e. Navigation, Multimedia, Telematics/Call Assistant Portable Device and other ECUs are connected to IVI

5 How We collaborate with Tizen Toyota developed reference platform based on Tizen IVI Released SW packages to tizen.org include Weston plugin Device input controller Pulse Audio plugin Vehicle information control (VIC) plugin for automotive message broker Car simulator Sample home screen Sample apps Reference Platform Ver0.5 (RPF0.5) Connecting Car, Driver and Internet ico

6 TIZEN IVI RPF0.5 on Tizen IVI RPF0,5 OSS RPF0,5 sample code Closed( Proprietary etc.) Portable Device IVI App Layer ECU Apps HomeScreen Native Apps HTML5 Apps Native App Tizen-IVI EFL Tizen-IVI EFL UI-component Webkit-EFL Remote UI Remote UI Remote UI Wayland/Weston Pulse Audio UI Layer weston -plugin pulse -plugin System Layer Murphy AMB Policy DB amb -plugin CorePF Layer Device Input Controller Cgroups Smack Car Simulator OEM CAN MW internet Tizen IVI on Nexcom NDiS 166

7 UI Manager Multi Window Manager Multi Input Manager Multi Sound Manager Vehicle Information Control Home Screen

8 UI-Manager Focus Requirements Easy-to-use user interface Simple and Smart architecture to reduce SW complexity Robustness under limited memory resource on embedded CPU Automotive reliability Mobile Apps portability Toyota UI-Manager Utilize OSS community efforts and common code base Adapt software layer architecture Use plugin and common IPC Support mechanism to avoid driver distraction

9 Role of UI-Manager IVI Remote-UI UI-Manager (View) Multi Sound Mgr. For Audio Tel/TTS Multi Input Mgr. For touch SW steering SW STT Multi Window Mgr. Center Display Window Mgr. MID/Cluster Display Tizen-IVI Apps (Control or Model) Portable Device Proxy Navigation Apps Home Screen Radio TV Air-conditioner Camera Cluster ECU (Model) Portable device Audio Air Conditioner Cluster

10 UI Manager Multi Window Manager Multi Input Manager Multi Sound Manager Vehicle Information Control Home Screen

11 MWM Usage example with multi layers Service Apps Data Type Cluster ECU Camera Back Camera Front Camera MID/Cluster Informatio n Super imposed Menu Apps AV 3D 2D 3D RGB Wayland + UI-Manager Center display Remote U/I 2D 3D Navigation Route guidance 2D 3D remote UI (Drawing API) Map 3D 2D IVI Portable Device

12 MWM functional requirements Sophisticated user interface Multi layers for IVI High performance - Wayland/Weston plugin High scalability - Pluggable OEM specific drawing App coexistence with car OEM applications Monitor Window operation of applications Apps execution environment (Web, Native)

13 Why do we want to use Wayland/Weston? Simple and Smart Wayland/Weston = Display Server + Window Manager Direct rendering No use of X11 rendering protocol High scalability Easy to add OEM original features Drawing method 3D animation, etc Car OEM original UI devices (Haptic device, etc)

14 MWM feature(rpf0.5) Basic window functions Create and delete two or more windows Manage it's position, width and height, visible/invisible and effects Manage two or more windows within one display device Window Layer Management Free arrangement of the layers Batch control of windows on the layers Window Layout Management (2 types) Compositing window manager using Normal Apps, Pop-Up and so on. Tiling window manager using Home-Screen and so on

15 MWM software block diagram (RPF0.5) HomeScreen Native App HTML5 App UX-FW(API) UI Component Webkit2-EFL OpenGL/EGL API EFL Wayland API Weston Application Mgr App Manifest App Manifest App Manifest Configurations Wayland-Server IVI Shell Plugin Loader IVI Common RemoteGL Proxy Multi Window Manager Multi Input Manager Remote UI Broker Other ECU Weston 別 ECU OpenGL ESv2/EGL Function call WebSocket Protocol Wayland Protocol

16 UI Manager Multi Window Manager Multi Input Manager Multi Sound Manager Vehicle Information Control Home Screen

17 MIM functional requirements Sophisticated user interface High efficiency - Weston plug-in High scalability - Can add OEM specific input device Manage input events Abstract Input Event Map Input Event and each application Arbitrate Input Event conflict Suppress Input Event by vehicle condition Near future, MIM manages the input data even for other ECU

18 MIM feature (RPF0.5) Basic Input Device Mouse (Touch-Screen), Keyboard, Joystick Mapping Apps and Input devices Mapping by rule-based DB Deliver the event to the apps Support Car OEM specific devices Device Input Controller

19 MIM software block diagram (RPF0.5) HomeScreen UX-FW(API) Native App HTML5 App UI Component OpenGL/EGL API Wayland API Weston Wayland-Server IVI Shell Plugin Loader IVI Common RemoteGL Proxy Multi Window Manager Multi Input Manager Remote UI Broker WebSkeleton Webkit2-EFL EFL Other ECU Weston 別 ECU Function call WebSocket Protocol Wayland Protocol Device Input Controllers Remote リモートタッチ touch Haptic ハプテック HW スイッチ switch 入 力 Touch タッチ input 入 力

20 UI Manager Multi Window Manager Multi Input Manager Multi Sound Manager Vehicle Information Control Home Screen

21 MSM functional requirements Sophisticated user interface High efficiency - pulse audio plugin High scalability - Pluggable Car OEM specific sound functions App Coexistence with car OEM apps Apps execution environment (Web, Native) Monitoring and Arbitration sound operation(*) of each application (*) Mixing/attenuation/pause/Mute/Cut off etc. Zone-control Multi sound zone (front and rear etc.) Near future, Remote-UI MSM manages the sound (in/out) even for other ECU

22 MSM feature (RPF0.5) Basic Audio functions Dynamic Volume Control Dynamic Mixing Control User operation (Audio apps, Mute button etc.) Other situation (Phone call etc.) Manage apps using Sound Arbitrate sound operation from each application (Priority scheduling by rule-based DB) Cooperation with the Home screen Mute, Cancel, get attribute notify status change to application

23 MSM software block diagram (RPF0.5) HomeScreen Native App HTML5 App libpulse In/out WebSkeleton control In/out Webkit2-EFL EFL libpulse In/out Other ECU PulseAudio Hook Pulse Audio UIFW-PlugIn Hook Rule-based DB out in control ALSA out PulseAudio Protocol WebSocket Protocol Wayland Protocol

24 UI Manager Multi Window Manager Multi Input Manager Multi Sound Manager Vehicle Information Control Home Screen

25 VIC functional requirement VIC caches vehicle data sent by the ECU. Vehicle data is offered by the demand from the Vehicle apps. VIC uploads vehicle data to the center. Vehicle apps controls each ECU. 3 Gate Way Vehicle apps Automotive BUS 4 2 Vehicle Information control 1 IVI

26 VIC feature (RPF0.5) High efficiency Work as Automotive Message Broker plug-in Manage Vehicle Data Abstract vehicle data Data format for in-vehicle LANs may change Two Data format type Real-time data in the cache History data in the vehicle-information-db Two API property type Common API OEM specific API Non Disclosure Car simulator for software debug Can simulate the running state of the vehicle Steering, Acceleration pedal, Brake Pedal etc. Increase efficiency for debugging during vehicle test API for car simulator and vehicle middleware are the same

27 VIC Software Block Diagram (RPF0.5) Native HTML5 HTML5 Apps Apps Apps Apps Apps Apps Apps Apps Apps Probe Manager D-Bus plugin AMB WebSocket sink Other ECU AMB User info User Manager Device info History Weston Vehicle info History (standard) Vehicle info History (OEM) History (standard) AMB core History (OEM) OEM VIC API VIC plugin Standard VIC API WebSocket Source VIC MW Car Simulation TIZEN RPF0.5 Car Drv CAN HW WebSocket socket CAN 27

28 VIC topics Policy of API Definitions Contents Provider needs open standard vehicle API Car OEM can not support all standard API Depend on each car OEM Personal information protection policy Safety, Security policy Business strategy (probe data property etc.) Depend on each vehicle model Equipment of vehicle package option (Navigation, ADAS, etc) Automotive LAN grade (e.g. not connected some ECU) We need to define open standard API! But, We should be provided with the access control every API (Property)

29 UI Manager Multi Window Manager Multi Input Manager Multi Sound Manager Vehicle Information Control Home Screen

30 Home Screen feature (RPF0.5) User Menu Simple Tile Menu (using Window Manager) Status Bar Launcher (with sample Apps) On Screen Application control regarding window, input event, sound Interrupt Action Change View for Driver Distractio (interrupted by vehicle inf. via Automotive Message Broker)

31 Overview of Sample Home Screen Local Apps Navigation Audio System Controller Home Screen Native Apps Web Apps API G-book Entune Navigation Audio UI manager EFL HTML5 POSIX System call (Process/File/device) PF Navigation G-Book Entune Tizen IVI (with UI Manager +VIC) Linux kernel Navi core: maps, route search and guidance

32 Home Screen & System Controller Home Screen System Controller Layout Policy DB each Destination/Vehicle OEM Home Screen Main Menu Status Bar On Screen Apps Controller Resource control Life cycle (launch,) Access control Multi User Device Controller Vehicle Controller Native Apps Request resource HTML Apps AppsFW lib Apps Framework PF lib UX-Framework Tizen API EFL AMB PulseAudio Murphy etc.

33 HS topics Sample Screen Layers (RPF0.5) Event handling (Input Layer) On Screen Message (ONS layer) Apps Display (Apps Layer) Apps Menu (Home Screen Layer) status bar, live thumbnail, Apps launcher etc. Input ONS APPS HS Multi Layer Apps Menu (Tile Window) RPF0.5 sample code

34 HS topics Change View for Driver Distraction In IVI system, application must follow the rule and the law. Only trusted applications are allowed to display during driving. UI-Component is libraries for trusted application. Home Screen (or alternate system app) Make invisible during driving No control even if driving Un-trusted Apps Trusted Apps Reduce development cost and time by using UI-Component. UI Component

35 UI Manager Multi Window Manager Multi Input Manager Multi Sound Manager Vehicle Information Control Home Screen ico

36 Download ivi-wayland/ Packages ico-vic-amb-plugin: Automotive Message Broker (AMB) plugin for delivering vehicle information to applications. ico-uxf-weston-plugin: Weston compositor plugin for UI management. ico-uxf-device-input-controller: Touch screen and USB steering wheel input device calibration. ico-uxf-homescreen-sample-apps: a set of sample applications from Toyota including AR navigation, sound and vehicle information controller applications. These apps are examples of how applications behave when the vehicle state changes. i.e. running, stop ico-vic-carsimulator: CAN bus emulator ico-uxf-pulse-plugin: a policy enforcement plugin to Pulse Audio ico-uxf-homescreen: sample UI for application launcher and manager. 36

37 Summary Toyota delivered OSS packages UI-Manager Vehicle Information Control Sample home screen Keep up to date on Tizen IVI Let s collaborate in OSS community

38 Any questions? Thank you!!

39 Backup 39

40 MWM TOPICS sequence chart App Home Screen OpenGL /EGL API Wayland API Wayland Server IVI Shell Weston Multi Window Manager Weston Compositor Multi Input Manager OpenGL/EGL Create Wayland Surface HomeScree WebSocket Protocol Wayland Protocol Create Shell Surface Shell Surface Create EGL Surface EGL Surface EGL Surface EGL Surface EGL Surface Positioresize - layer Shell Surface EGL Surface HomeScree Notify Window Resize surface Notify Window PositioSize Layer Window HomeScree Westo ( ) ( ivi-shell ) (HomeScree )

41 MWM TOPICS sequence chart App Home Screen OpenGL /EGL API Wayland API Wayland Server IVI Shell Weston Multi Window Manager Weston Compositor Multi Input Manager OpenGL/EGL Surface Surface WebSocket Protocol Wayland Protocol Surface Window Surface Surface EGL Surface HomeScree Surface Surface Surface Surface Surface Surface Window Surface HomeScree

42 MIM sequence chart (RPF0.5) App App App HomeScreen Wayland API Wayland Weston Multi Window Manager Multi Input Manager Device Input Controllers Remote Touch Touch Panel Steering Switch Input Devices Surface Surface ID WebSocket Protocol Wayland Protocol

43 MSM sequence chart (RPF0.5) App Home Screen Pulse Audio Pulse Audio UIFW-PlugIn ALSA Drv (A) (A ) ( ) A / / / / A A B / A A A ( ) B PID (A) ( ) ( ) ( / ) (B) ( / ) (A ) ( ) ( ) WebSocket Protocol Wayland Protocol

44 MWM TOPICS Application monitoring HomeScreen Native Apps HTML5 Apps App-FW Tizen-IVI EFL Tizen-IVI EFL UIcomponent Webkit-EFL Multi Window Manager Wayland/Weston

45 HS topics Change View for Driver Distraction Home Screen (or alternate system app) UI Component for Web Apps for Native Apps Murphy Automotive Message Broker VIC plugin. Vehicle Information If you want to use policy based decision making, "Murphy" is useful for it. CAN

46 HS topics UI Components functional overview Text labels Radio button Check box Tab panel Audio setting panel Element type Driving mode visibility Allowed with restriction below: - (Font size > limit) && (Text string length < limit) - If Text label supports text-scrolling, then scrolling is not allowed. (should stop scrolling) Animation (MNG, GIFanim ) Slider Accordion list Anchor and mode transition Text Input form Dropdown list box Date picker Allowed with restriction below: - Animation should stop. (only still image is allowed) Allowed with restriction below: - 2 value slider is not allowed, only 1 value. - Accordion should not be animated. Not allowed.