Seventh IEEE Workshop on Embedded Computer Vision. Ego-Motion Compensated Face Detection on a Mobile Device

Transcription

1 Seventh IEEE Workshop on Embedded Computer Vision Ego-Motion Compensated Face Detection on a Mobile Device Björn Scheuermann, Arne Ehlers, Hamon Riazy, Florian Baumann, Bodo Rosenhahn Leibniz Universität Hannover, Germany 1

2 Outline Motivation Boosting Based Face Detection Inertial Measurement Unit Proposed Method Experimental Results Conclusion 2

3 Viola and Jones Face Detection P.Viola and M.Jones. Rapid object detection using a boosted cascade of simple features. CVPR,

4 Viola and Jones Face Detection well known appearance-based approach utilizes Adaboost in the classifier training weak classifiers based on Haar-like features integral image for efficient feature computation standard in current commercial cameras P.Viola and M.Jones. Rapid object detection using a boosted cascade of simple features. CVPR,

5 Viola and Jones Face Detection well known appearance-based approach utilizes Adaboost in the classifier training weak classifiers based on Haar-like features integral image for efficient feature computation standard in current commercial cameras pros: general object detection framework good detection rates fast detection drawbacks: slow training highly rotational variant P.Viola and M.Jones. Rapid object detection using a boosted cascade of simple features. CVPR,

6 Motivation Viola and Jones Face detection: 4

7 Motivation Viola and Jones Face detection: 4

8 Motivation Viola and Jones Face detection: 4

9 Motivation Viola and Jones Face detection: 4

10 Motivation Viola and Jones Face detection: 4

11 Motivation Rotation invariant detection based on Viola and Jones? Du et al.: set of rotated Haar like features Wu et al.: divide human faces according to different view points 5

12 Motivation Rotation invariant detection based on Viola and Jones? Du et al.: set of rotated Haar like features Wu et al.: divide human faces according to different view points Computational too expensive for mobile devices (8-12 times slower) 5

13 Motivation Rotation invariant detection based on Viola and Jones? Du et al.: set of rotated Haar like features Wu et al.: divide human faces according to different view points Computational too expensive for mobile devices (8-12 times slower) Inertial sensors getting major attention markerless human motion capturing structure from motion... Inertial sensors are available on most current mobile devices mobile phones (iphone, HTC Desire,...) ipod touch, ipad,... 5

14 Motivation Viola and Jones face detection framework (rotational variant) 6

15 Motivation Viola and Jones face detection framework (rotational variant) mobile devices with an IMU (inertial measurement unit) + 6

16 Motivation Viola and Jones face detection framework (rotational variant) mobile devices with an IMU (inertial measurement unit) fusion?! + 6

17 Boosting Based Face Detection Adaboost is an algorithm for constructing a strong classifier as a linear combination of simple weak classifiers h t (x) : T f(x) = α t h t (x) t=1 final strong classifier/hypothesis: H(x) = sign(f(x)) 7

18 Haar-like Features Easy computation (scalable and changeable in size and position) Fast and efficient computation with integral-image I Σ (x, y) = i x i=0 j y j=0 I(i, j) A + D (C + B) 8

19 Example 9

20 Inertial Measurement Unit iphone 4, 4G ipod Touch, ipad 2 two hardware sources: three axis accelerometer three axis gyroscope 10

21 Inertial Measurement Unit World-coordinate System (defined by the IMU) spanned by the axes: (x world,y world,z world ) z world := a gravitation a gravitation x world,y world defined orthogonal to gravity (right-hand space) 11

22 Inertial Measurement Unit World-coordinate System (defined by the IMU) spanned by the axes: (x world,y world,z world ) z world := a gravitation a gravitation x world,y world defined orthogonal to gravity (right-hand space) Apple provides a proprietary algorithm to calculate absolute orientations relative to the static world-coordinate system. 11

23 Proposed Method Camera-coordinate system: We are interested in the rotation around the z-axis (yaw-angle) y Device z Device x Device 12

24 Proposed Method y Device z Device x Device z Proj z World z Device ' Yaw 13

25 Proposed Method Apples proprietary algorithm gives: y Device (0,i device,t )=q(0,i world )q 1 (i R 3 ) z Device x Device z Proj z World z Device ' Yaw 13

26 Proposed Method Apples proprietary algorithm gives: (0,i device,t )=q(0,i world )q 1 (i R 3 ) y Device z Device x Device project z world on the devices screen surface: z proj,t = z world z world z device,t z device,t z Proj z World z Device ' Yaw 13

27 Proposed Method Apples proprietary algorithm gives: (0,i device,t )=q(0,i world )q 1 (i R 3 ) y Device z Device x Device project z world on the devices screen surface: z proj,t = z world z world z device,t z device,t z Proj z World z Device compute the yaw-angle: cos(ϕ yaw )=± y device,t z proj,t / z proj,t ' Yaw having the yaw-angle, we rotate the camera image to produce a virtual upright image 13

28 Proposed Method Camera coordinate system z world project on the device screen surface compute the yaw-angle y Device z Proj z World ' Yaw z Device x Device z Device 14

29 Standard Approach image comming from mobile device Viola + Jones face detection 15

30 Proposed Method image comming from mobile device IMU compensate ego-motion compute yaw-angle generate virtual upright image Viola + Jones face detection 16

31 Proposed Method 17

32 Proposed Method 17

33 Proposed Method 17

34 Proposed Method 17

35 tated camera and upright face. The proposed fusion outperformed the standard algorithm. Experimental Results C AMERA FACE Sequence 1 straight straight Sequence 2 straight rotated Sequence 3 rotated straight SYMBOLIC Table 1: Evaluation of test cases Sequence 1, as shown in Figure 6a, analyzes whether the alteration and extension of the framework has a negative impact for the standard case. This is a situation of ideal conditions for the unmodified face detector. The face is almost perfectly aligned upright and thus corresponds to the expected value of the algorithm. The yaw-angle computed is almost zero, so that the individual frames are rotated with a small angle. The experimental video sequence consists of 165 frames and yaw-angles between 918 and 11. Figure 8: Detection-rate on vid is almost perfectly aligned uprig left and to the right. Both algor The y-axis states weather a fac The x-axis depicts the yaw-angl quality compared to the conven methods detect the face in every Similar to the first sequence rection of the camera orientatio however, turns from the ideal p expected and shown in figure recognize the face after a certai the camera was not perfectly ali performed slightly better. The e consists of 352 frames, a camera and 8 and a face rotation aro and 45. The graph in Figure 8 sh achieve a similar performance i The third sequence analyzes was the motivation for this pap of the camera and the camera b

36 tated camera and upright face. The proposed fusion outperformed the standard algorithm. Experimental Results C AMERA FACE Sequence 1 straight straight Sequence 2 straight rotated Sequence 3 rotated straight SYMBOLIC Table 1: Evaluation of test cases Sequence 1, as shown in Figure 6a, analyzes whether the alteration and extension of the framework has a negative impact for the standard case. This is a situation of ideal conditions for the unmodified face detector. The face is almost perfectly aligned upright and thus corresponds to the expected value of the algorithm. The yaw-angle computed is almost zero, so that the individual frames are rotated with a small angle. The experimental video sequence consists of 165 frames and yaw-angles between 918 and 11. Figure 8: Detection-rate on vid is almost perfectly aligned uprig left and to the right. Both algor The y-axis states weather a fac The x-axis depicts the yaw-angl quality compared to the conven methods detect the face in every Similar to the first sequence rection of the camera orientatio however, turns from the ideal p expected and shown in figure recognize the face after a certai the camera was not perfectly ali performed slightly better. The e consists of 352 frames, a camera and 8 and a face rotation aro and 45. The graph in Figure 8 sh achieve a similar performance i The third sequence analyzes was the motivation for this pap of the camera and the camera b

37 tated camera and upright face. The proposed fusion outperformed the standard algorithm. Experimental Results C AMERA FACE Sequence 1 straight straight Sequence 2 straight rotated Sequence 3 rotated straight SYMBOLIC Table 1: Evaluation of test cases Sequence 1, as shown in Figure 6a, analyzes whether the alteration and extension of the framework has a negative impact for the standard case. This is a situation of ideal conditions for the unmodified face detector. The face is almost perfectly aligned upright and thus corresponds to the expected value of the algorithm. The yaw-angle computed is almost zero, so that the individual frames are rotated with a small angle. The experimental video sequence consists of 165 frames and yaw-angles between 918 and 11. Figure 8: Detection-rate on vid is almost perfectly aligned uprig left and to the right. Both algor The y-axis states weather a fac The x-axis depicts the yaw-angl quality compared to the conven methods detect the face in every Similar to the first sequence rection of the camera orientatio however, turns from the ideal p expected and shown in figure recognize the face after a certai the camera was not perfectly ali performed slightly better. The e consists of 352 frames, a camera and 8 and a face rotation aro and 45. The graph in Figure 8 sh achieve a similar performance i The third sequence analyzes was the motivation for this pap of the camera and the camera b

38 tated camera and upright face. The proposed fusion outperformed the standard algorithm. Experimental Results C AMERA FACE Sequence 1 straight straight Sequence 2 straight rotated Sequence 3 rotated straight SYMBOLIC Table 1: Evaluation of test cases Sequence 1, as shown in Figure 6a, analyzes whether the alteration and extension of the framework has a negative impact for the standard case. This is a situation of ideal conditions for the unmodified face detector. The face is almost perfectly aligned upright and thus corresponds to the expected value of the algorithm. The yaw-angle computed is almost zero, so that the individual frames are rotated with a small angle. The experimental video sequence consists of 165 frames and yaw-angles between 918 and 11. Figure 8: Detection-rate on vid is almost perfectly aligned uprig left and to the right. Both algor The y-axis states weather a fac The x-axis depicts the yaw-angl quality compared to the conven methods detect the face in every Similar to the first sequence rection of the camera orientatio however, turns from the ideal p expected and shown in figure recognize the face after a certai the camera was not perfectly ali performed slightly better. The e consists of 352 frames, a camera and 8 and a face rotation aro and 45. The graph in Figure 8 sh achieve a similar performance i The third sequence analyzes was the motivation for this pap of the camera and the camera b

39 Conclusion proposed an efficient fusion of IMU and boosting based face detection by Viola and Jones 19

40 Conclusion proposed an efficient fusion of IMU and boosting based face detection by Viola and Jones compared to other (rotation invariant) boosting based method our method is fast enough for a mobile device 19

41 Conclusion proposed an efficient fusion of IMU and boosting based face detection by Viola and Jones compared to other (rotation invariant) boosting based method our method is fast enough for a mobile device video sequences + motion data will be available 19

42 Conclusion proposed an efficient fusion of IMU and boosting based face detection by Viola and Jones compared to other (rotation invariant) boosting based method our method is fast enough for a mobile device video sequences + motion data will be available online demo is available (just ask) 19

43 Conclusion proposed an efficient fusion of IMU and boosting based face detection by Viola and Jones compared to other (rotation invariant) boosting based method our method is fast enough for a mobile device video sequences + motion data will be available online demo is available (just ask) compensate face motion is an ongoing project 19

44 Conclusion proposed an efficient fusion of IMU and boosting based face detection by Viola and Jones compared to other (rotation invariant) boosting based method our method is fast enough for a mobile device video sequences + motion data will be available online demo is available (just ask) compensate face motion is an ongoing project Thanks for your attention!!! 19