App analytics: evaluating the distraction potential of in-vehicle device apps

Transcription

1 App analytics: evaluating the distraction potential of in-vehicle device apps Institute of Ergonomics (TU Munich) Carmeq Daimler Laudenklos Engineering GM Corporation Michael Krause, Antonia Conti, Klaus Bengler Matthias Henning, Christopher Seubert Christian Heinrich Carolin Herrigel Daniel Glaser funded by the Car Connectivity Consortium (CCC)

2 Agenda Motivation Methods Results Discussion 30th July 2015 Institute of Ergonomics - 2

3 Project Purpose in a Nutshell Investigate different subtasks used in mobile device apps. Subtask examples: scroll a list, enter a zip code Typical subtasks will be evaluated to explore their distraction potential. Main project goal is to be able to judge whether an app is ready to be tested in a driving simulator (with test participants) based on its component subtasks. 30th July 2015 Institute of Ergonomics - 3

4 Approach & Aim Acquire gaze duration and occlusion values (used to quantify driver distraction) for typical interactions (e.g. scroll a list, type a name, etc.) with natural apps while driving in a mockup. Aim: to evaluate the possibility and feasibility of a method for human factors experts to predict the experimental outcome of a driving simulation experiment so clearly unsuitable apps can be pre-excluded from further testing. 30th July 2015 Institute of Ergonomics - 4

5 Other Task Analytics Task Analysis GOMS Model (Goals, Operators, Methods, Selection rules) Keystroke-level (KLM-GOMS) SAE J 2365 Calculation of the Time to Complete In-Vehicle Navigation and Route Guidance Tasks» Adaptation of KLM-GOMS to In-Vehicle Tasks» 15-Second Rule Methods Time Measurement 30th July 2015 Institute of Ergonomics - 5

6 Overview First Experiment: Acquire data while driving and interacting with touch screen apps Second Experiment: Acquire data while driving and interacting with a rotary knob Predict the outcomes (Single Glance Duration, Total Glance Time and Total Shutter Open Time) of the third experiment with the outcomes the first two experiments Third experiment: Acquire data while driving and interacting with touchscreen and rotary knob apps Compare predictions and outcomes of third experiment 30th July 2015 Institute of Ergonomics - 6

7 Methods

8 Measurement Methods Occlusion Test person performs a task with glasses on that periodically open (e.g. transparent) and close (e.g. opaque). Task interruptability and resumability are tested. Open/closed intervals used: open: 1500; closed: 1000 ms Eye tracking Test person wears Dikablis eye tracking headset. Single glance durations and total glance times are measured. 30th July 2015 Institute of Ergonomics - 8

9 Simulated Driving Task AAM scenario adapted to German Autobahn standards Participants were instructed and trained to follow a leading vehicle traveling at 80km/h at a distance of approximately 50m Seating mockup with SILAB 4.0 (WIVW GmbH, Würzburg) 30th July 2015 Institute of Ergonomics - 9

10 Touch Tablet Intenso Tab 824, rooted and screen adjusted to display at: 800 x ppi (5.8 ). 30th July 2015 Institute of Ergonomics - 10

11 Rotary & Radio Rotary: Daimler COMAND system with iphone coupling (Digital DriveStyle) Radio: Sony CDX-GT570UI 30th July 2015 Institute of Ergonomics - 11

12 Procedure Approximate experiment duration: Experiment 1 (touch): 3-4 hrs Experiment 2 (rotary knob): 2 hrs Experiment 3 (both): 2 hrs Procedure: Participants were trained with the system(s) under investigation. Different system tasks performed with occlusion and eye tracking in randomized order. Each task was separately explained and trained, followed by two measurement blocks (occlusion and eye tracking). Order of occlusion and eye tracking was also randomized. Reference tasks were performed: Radio tuning task. 30th July 2015 Institute of Ergonomics - 12

13 Example Slider Four subtasks Select icon (start app) System delay Swipe and select button Adjust slider 30th July 2015 Institute of Ergonomics - 13

14 Metrics of Interest Single Glance Duration Sum of sequential fixations to an Area of Interest (AoI) Total Glance Time (Eye tracking) Sum of single glance durations to an Area of Interest (AoI) during a task. Total Shutter Open Time (Occlusion) Sum of vision intervals i.e. total time the interface was visible 30th July 2015 Institute of Ergonomics - 14

15 85 th Percentile The 85 th Percentile (P85) is often used for gaze metrics. P85 tells that 85% of all measurements are below this P85 value. 30th July 2015 Institute of Ergonomics - 15

16 Berlin-Munich-Method Instead of storing a single (mean) value (Y) for every sub-task, we store a (mean) value (Y1, Y2, Y3, Yn) for each subject and every subtask - by doing this, we run a "virtual experiment" Data base Subject_1 Subject_2 Subject_3 Sub-task_1 2s 1.5s 3s Sub-task_2 4s 3s 3s Sub-task Virtual Experiment Subject_1 Subject_2 Subject_3 Sub-task_7 5s 4s 6s Sub-task_9 3s 2.5s 2s Result 8s 6.5s 8s 30th July 2015 Institute of Ergonomics - 16

17 Tasks in evaluation experiment 3 Short Name Device Count of Subtasks Short Description T1 Touch 8 Enable a checkbox in a configuration submenu of an app and leave application T2 Touch 2 Enter a calculation into calculator (about 10 input steps) T3 Touch 5 Record a short voice message/note (one word) T4 Touch 7 Search radio stream by text search (6 chars) R1 Rotary 9 Share your location (Glympse) with someone from the contact list R2 Rotary 10 Reply to a message with a predefined short text R3 Rotary 3 Switch off the infotainment screen R4 Rotary 5 Search a radio stream by text search (2 chars) 30th July 2015 Institute of Ergonomics - 17

18 Participants Experiment N male /female Age min max; M (SD) Left handed Eye ware needed <10,000 km/year >20,000 km/year / ; 56 (6) / ; 59 (6) / ; 55 (4) Subjects from the first two experiments were not allowed to participate in the third experiment. 30th July 2015 Institute of Ergonomics - 18

19 Results

20 Total Glance Time 85th Percentiles (TGT) 25 estimated measured -7% 20 19% 3% -5% Seconds % 1% 36% 5 102% 0 T1 T2 T3 T4 R1 R2 R3 R4 30th July 2015 Institute of Ergonomics - 20

21 Seconds Total Shutter Open Time 85th Percentiles (TSOT) 25 estimated measured 20 7% 6% 6% -8% 15-6% 10 27% -12% 5 94% 0 T1 T2 T3 T4 R1 R2 R3 R4 30th July 2015 Institute of Ergonomics - 21

22 Single Glance Duration 85th Percentiles (SGD) 2,0 1,8 12% 8% estimated measured 1,6 15% 0% 1,4 28% -4% -7% 6% Seconds 1,2 1,0 0,8 0,6 0,4 0,2 0,0 T1 T2 T3 T4 R1 R2 R3 R4 30th July 2015 Institute of Ergonomics - 22

23 Discussion

24 Discussion / Conclussion Results can be used in two ways: 1. The list of subtasks with characteristic values (TGT, TSOT, SGD) could be a low-cost, valuable tool for developers looking to include different subtasks in their app and to estimate their distraction potential. It can be useful to make them more aware of driver distraction issues, thus, an educational aspect. 2. Virtual experiment: determining subcomponents of a task is somewhat subjective, it is often difficult to find an appropriate subtask (assuming it has already been tested).further work is needed. But might be able to filter out clearly unreasonable and ill-suited apps before subject testing. 30th July 2015 Institute of Ergonomics - 24

25 App analytics: evaluating the distraction potential of in-vehicle device apps Thanks for your attention! Michael Krause Graduate Research Associate Faculty of Mechanical Engineering Institute of Ergonomics Boltzmannstraße 15 D Garching Tel Fax th July 2015 Institute of Ergonomics - 25