Methodology of evaluating the driver's attention and vigilance level in an automobile transportation using intelligent sensor architecture and fuzzy logic A.Giusti, C.Zocchi, A.Adami, F.Scaramellini, A.Rovetta Politecnico di Milano Robotics Laboratory IFTOMM 2007
Index 1. Identification and choice of the parameters 2. Realization of the experimental system 3. Driving sessions 4. Off-line analysis 5. Results 6. Construction and validation of the fuzzy classifier 7. The final control system
1. Identification and choice of the parameters Blood pressure Breath frequency Reaction time Eye movement Head inclination Heart rate variability (HRV) Galvanic skin response (GSR) Skin temperature (THE) Non-invasive Chosen parameters Cerebral waves Reference parameter
2. Realization of the experimental system Medical signals acquisition system Centro del Sonno - Fondazione Maugeri Driving simulator Keplero University of Linz EEG sensors Measurement PC Steering wheel GSR, THE, HRV acquisition Politecnico di Milano Mobile platform actuators Controller Gas GSR,THE, HRV sensors Measurement PC Brake Circuit Vigilance level
2. Realization of the experimental system GSR, THE, HRV acquisition system - Politecnico di Milano GSR HRV THE
2. Realization of the experimental system Medical signals acquisition system Centro del Sonno Fondazione Maugeri
2. Realization of the experimental system Driving simulator Keplero University of Linz
2. Realization of the experimental system Driving simulator Keplero University of Linz
3. Driving sessions Number of subjects Number of tests Number of repetitions Duration Acquired parameters Circuit protocol 20 5 for 4 types of circuit 4 2 min HRV, GSR, THE Vigilance protocol 16 1 test for 2 levels of vigilance (sleepy or not) 2 26 min HRV, GSR, THE, EEG
4. Off-line analysis (acquisition and pre-processing) Politecnico di Milano Signal acquisition Signal pre-processing Signal to be processed MATLAB NI-DAQ card 6062E fc 200Hz FFT analysis Filter design Extraction of parameters Fondazione Maugeri Signal acquisition Signal pre-processing Signal to be processed fc 200Hz Somnologica Automatic filtering Extraction of parameters
4. Off-line analysis (signal elaboration) Politecnico di Milano EEG PSD analysis Fondazione Maugeri Exact microsleep moment EEG normal analysis Two types of analyses: I) General behavior of the signals towards sleepiness II) The minute before the microsleep
4. Off-line analysis (signal elaboration) I) General behavior of the signals towards sleepiness Classical statistics Multivariate statistics Comparison sleepy/ non-sleepy situations II) The minute before the microsleep Classical statistics Multivariate statistics Division in 10s intervals before the microsleep Comparison interval per interval
5. Results I) General behavior of the signals towards sleepiness THE and Heart Frequency tend to drop. 1. The drop of temperature and Heart rate can be sleepiness precursors 2. The drop is usually of less than 1 degree for THE 3. The whole procedure is quite slow for THE and quite general for the Heart rate GSR tends to raise 1. The GSR raise can be an indicator of sleepiness and/ or attention level drop 2. The raise can be quite important (*10) 3. The procedure is faster, but still quite slow
5. Results II) The minute before the microsleep Important variations in the Heart rate 1. The heart rate generally tends to drop but the most important thing noticed is that it presents some significant variations several seconds before the microsleep 2. At least in 76% of the cases these variations were present. The percentage can improve using a lower threshold value 3. The procedure is very fast and so the microsleep could be recognized in time
5. Results II) The minute before the microsleep Very small standard deviation values for the error in the car s position 1. The error the driver is making was calculated as the difference from the ideal position of the car on the road 2. Each driver has his/her own driving style and so his/her own mean error values. The data were normalized using these values 3. The important thing noticed is that during the minute before the microsleep the standard deviation value of the error made is much lower that usual. Meaning that, even if the driver is driving far from the ideal position he is not moving the wheel as usually 4. This phenomenon was observed in 87,5% of the cases and can be augmented by lowering the threshold value by little
6. Construction and validation of the fuzzy classifier GSR, THE, HRV Data analysis EEG Fuzzyfication MEMBERSHIP FUNCTION Implication RULES Defuzzyfication OUTPUT PARAMETER SS Data Information
6. Construction and validation of the fuzzy classifier Why Fuzzy logic? The classification of the level of vigilance is not deterministic, but qualitative In this way the medical experience could be directly integrated 1 complete vigilance level W 2 different logics 2 sleepiness levels S1 e S2 1 complete vigilance level W 1 sleepiness level S
6. Construction and validation of the fuzzy classifier 1 complete vigilance level 2 levels of sleepiness Reduced accuracy Better accuracy 1 complete vigilance level 1 level of sleepiness
7. The final control system Data acquisition of HRV, GSR, THE and steering wheel position Pre-processing Saving Fuzzy classifier Neural network HRV and steering wheel variations Alarm
Publications Methodology of evalueting safety in automobiles using intelligent sensor architecture and neural networks, Sensors and Actuators A Physical, 2007,134/2 pp. 622-630 Feasibility analysis on a car control system by psychic-physical parameters, International Journal of Mechanics and Control JoMaC, 2006, vol.02, No. 07 THANK YOU FOR YOUR ATTENTION Biorobotic system for increasing automotive safety, C.Zocchi, A.Giusti, A.Adami, F.Scaramellini, A.Rovetta, IFToMM 2007 World Congress Methodology of evaluating the driver's attention and vigilance level in an automobile transportation using intelligent sensor architecture and fuzzy logic, A.Giusti, C.Zocchi, A.Adami, F.Scaramellini, A.Rovetta, IFToMM 2007 World Congress