ICT 7 th, 2010, Tokyo A Robust And Efficient Face Tracking Kernel For Driver Inattention Monitoring System Yanchao Dong, Zhencheng Hu, Keiichi Uchimura and Nobuki Murayama Kumamoto University, Japan
Kumamoto, Japan
Our Lab ITS Research Lab, Kumamoto University http://navi.cs.kumamoto-u.ac.jp More than 30 members working on Computer vision based ITS applications Direct vision navigation Sensor fusion for Full Speed Range ACC Occupant detection for airbag development control Intelligent traffic surveillance system Satellite image processing for digital map Human navigation Optimization techniques for ITS Dynamic Route Planning Conveyor Traffic Control
Outline 1. Introduction 2. Current Methods to Detect Driver Inattention 3. Face Trackers 4. Zero-order binocular EKF face tracker 5. Conclusion 熊 本 大 学 ITS 研 究 室
Introduction Motivate Driver Inattention Monitoring System Distribution of Inattention to accidents Data from NHTSA 2000[1] (Tekscan) Aims of this paper: review the current methods and try to make some analysis. (FaceLab) 熊 本 大 学 ITS 研 究 室
Introduction Two main categories of Driver Inattention Inattention monitoring Distraction Fatigue monitoring visual distraction cognitive distraction auditory distraction biomechanical distraction local physical fatigue general physical fatigue central nervous fatigue mental fatigue sleepiness Driver Inattention Monitoring System 熊 本 大 学 ITS 研 究 室
Current Methods to Detect Driver Inattention 1. Biological signal processing EEG UCL,DLDCN EOG ECG ADAM semg (noromed) Example of EEG signal (wikipedia) 2. Behaviour analysis Seat pressure (SP) Steering angle (SA) Pedal signal (PS) Lane position (LP) Speed signal (SS) (Tekscan) Following distance (FD) 熊 本 大 学 ITS 研 究 室
Current Methods to Detect Driver Inattention 3. Image Processing Based Approaches Two stages: Extract interesting parameters Estimate attention level based on the parameters 熊 本 大 学 ITS 研 究 室 the face position & orientation the eyeball yaw & pitch the eyelids animation the mouth & jaw animation.
Face Trackers Definition: A face tracking system estimates the rigid or nonrigid motion of a face through a sequence of image frames. Types of trackers Motion-based versus Model-based Appearance-based versus Feature-based
Zero-order order binocular EKF face tracker Model-based and Feature-based tracker Zero-order: faster, more stable and high passband Binocular: more accurate than monocular and could tell depth from scaling factors EKF: real-time with good accuracy Functioning principle Registered face model Tracking Kernal Measurement of the FPs in two camera image planes Tracking Result
Automatic face registration Face registration General face model Registration Kernel Real face Measurement of the FPs in two camera image planes Registered face
Automatic face registration Face registration: 10 shape parameters and 3 scaling factors are to be registered Coupling effect 1 unit/div rx ry rz tx ty tz eyesvdiff mouthwidth mouthvp nosepointingup nosevertical nosez eyeseparation eyeheight eyewidth eyesvp scaler cz scaler cy scaler cx Translation z and scaling factors can be estimated respectively 0 1 2 3 4 5 6 7 8 9 10 tiem/second Real value Estimated
Performance comparison of different kernels Performance of zero-, first- and second-order kernels on pose tracking Coupling effect rx ry Real value Zero-order First-order Second-order 1 unit/div rz tx ty tz 0 1 2 3 4 5 6 7 8 9 10 Timesecond tiem/second second
Performance comparison of different kernels Performance of monocular and binocular kernels on pose tracking Coupling effect FP B Real value Monocular Binocular FP M rx 1 unit/div ry rz tx ty tz 0 1 2 3 4 5 6 7 8 9 10 Time tiem/second
Pose and animation estimation Performance of zero-order binocular kernel on pose and animation tracking Coupling effect 1 unit/div rx ry rz tx ty tz upperlipraise lipstretcher jawdrop eyeballpitch eyeballyaw eyesclosed Real value Estimated 0 2 4 6 8 10 tiem/second Time
Robustness against model registration error SNR of model registration varies from 20 to 10 1 unit/div SNRmodel rx ry rz tx ty tz upperlipraise lipstretcher jawdrop eyeballpitch eyeballyaw eyesclosed Real value Estimated 0 2 4 6 8 10 tiem/second Time
Robustness against camera calibration error SNR of camera calibration varies from 40 to 20 1 unit/div SNRcalibration rx ry rz tx ty tz upperlipraise lipstretcher jawdrop eyeballpitch eyeballyaw eyesclosed Real value Estimated 0 2 4 6 8 10 tiem/second Time
Robustness against measurement noise SNR of camera calibration varies from 40 to 10 1 unit/div SNRmeasure rx ry rz tx ty tz upperlipraise lipstretcher jawdrop eyeballpitch eyeballyaw eyesclosed Real value Estimated 0 2 4 6 8 10 tiem/second Time
Thank you for your attention.
Current Methods to Detect Driver Inattention 3 main types of methods: Biological signal processing Behaviour analysis Image Processing Based Approaches Advantages Direct bio-signal measurement Signal acquisition has no interference on driver More driver information can be extracted Disadvantages Measurement needs contact the driver Signal analysis is not easy Processing algorithm is quite complicated 熊 本 大 学 ITS 研 究 室
Introduction Current Research Focus Image processing based approaches behavior analysis approaches 90% Fatigue Detection Bio-signal processing approaches behavior analysis approaches Image processing based approaches 10% Distraction Detection 熊 本 大 学 ITS 研 究 室