Faculté Polytechnique. Computational Attention and Social Signal Processing for Video Surveillance. Matei Mancas

Transcription

1 Faculté Polytechnique Computational Attention and Social Signal Processing for Video Surveillance Matei Mancas

2 Overview Attention? «Social» features Top-down models Camera embodiment Université de Mons Prof. Untel Service Untel (voir pied de page dans le menu Powerpoint) 2

3 Rough data Rare, Novel, Uncompressible, Surprising Goal, Models, Recognition Interesting data Bottom-up, Exogenous, Unsupervised Top-down, Endogenous, Supervised Memory, Emotions, Actions

4

5 Our approach Feature extraction Pre-processing Feature rarity/contrast Feature attention fusion Acquired signal Saliency map Image initiale Probabilité d intérêt

6 Features extraction On each pixel (slow ), on neighborhoods (more efficient) Static features (mean/standard deviations/social ) Dynamic features (speed/direction/contraction index/ ) 6

7 Pre-processing Spatio-temporal pyramid-like filtering (different object sizes) 7

8 Rarity/Contrast Bottom-up saliency: localy contrasted, globaly rare [Mancas, LNCS 2009]

9 Rarity/Contrast Still images 9

10 Rarity/Contrast Vidéos [Mancas, Riche, Leroy, Gosselin, ICIP 2011] 10

11 Rarity/Contrast Z motion and perspective issues: use of the Kinect [Riche, Mancas, Dutoit, submitted] Université de Mons Faculté Polytechnique de Mons 11

12 Rarity/Contrast Results Université de Mons Faculté Polytechnique de Mons 12

13 Overview Attention? «Social» features Top-down models Camera embodiment Université de Mons Prof. Untel Service Untel (voir pied de page dans le menu Powerpoint) 13

14 Social features Inter-personal spaces (proxemy) [Julien, Mancas, Gosselin, WIC 2011]

15 Caractéristiques sociales? Guessing people relations (and new guessing people intentions)

16 Attention in social features Based on features contrast(3 people) and feedback with HAL eyes Find the person who is different (no specific rules) Find someone who wants to avoid the others. [Julien et al INTETAIN 2011]

17 Overview Attention? «Social» features Top-down models Camera embodiment Université de Mons Prof. Untel Service Untel (voir pied de page dans le menu Powerpoint) 17

18 Top-down Toy example

19 Top-down [Mancas, Gosselin, BICS 2010] 19

20 Visitors styles in museums Top-down In open scenarios, styles are important

21 Overview Attention? «Social» features Top-down models Camera embodiment Université de Mons Prof. Untel Service Untel (voir pied de page dans le menu Powerpoint) 21

22 Camera embodiment Target visibility maximization Displacement minimization (effort) [Mancas, Pirri, Pizolli submitted]

23 Camera embodiment

24 Camera embodiment Coherent motion on X and Y -> SALIENT OBJECT

25 Keep in mind Bottom-up attention: choose within the object in motion Top-down models: learn normality on the long-term Depth information can enhance the analysis of small groups close to the camera Social features can be used to detect and predict the relation between people Camera embodiment could be used to delegate part of the computation load to the mechanical system (pan-tilt)

26 References [Mancas, LNCS 2009] M. MANCAS, 2009, "Relative Influence of Bottom-Up and Top-Down Attention", Attention in Cognitive Systems, Lecture Notes in Computer Science, Springer Berlin / Heidelberg, ISSN (Print) (Online), Volume 5395/2009, DOI / , ISBN [Mancas, Riche, Leroy, Gosselin, ICIP 2011] M. MANCAS, N. RICHE, J. LEROY, B. GOSSELIN, 2011, "ABNORMAL MOTION SELECTION IN CROWDS USING BOTTOM-UP SALIENCY", IEEE ICIP, Bruxelles, Belgium. [Mancas, Gosselin, BICS 2010] M. MANCAS, B. GOSSELIN, 2010, "Dense crowd analysis through bottom-up and top-down attention", Proc. of the Brain Inspired Cognitive Sytems (BICS 2010), Madrid, Spain. [Mancas, Couvreur, Gosselin, Macq, WCAA 2007] M. MANCAS, L. COUVREUR, B. GOSSELIN, B. MACQ, 2007, "Computational Attention for Event Detection", Proceedings of ICVS Workshop on Computational Attention & Applications (WCAA-2007), Bielefeld, Germany, Mar *Julien, Mancas, Gosselin, WIC J. Leroy, M. Mancas, B. Gosselin, Personal Space Augmented Reality Tool, First joint WIC/IEEE SP Symposium on Information Theory and Signal Processing in the Benelux, Bruxelles, Belgium, [Julien et al INTETAIN 2011], Julien Leroy, Nicolas Riche, François Zajega, Matei Mancas, Joelle Tilmanne, Bernard Gosselin, Thierry Dutoit, «The Attentive Machine: be Different!», INTETAIN 2011, Genoa, Italy