Automatic Maritime Surveillance with Visual Target Detection

Transcription

1 Automatic Maritime Surveillance with Visual Target Detection Domenico Bloisi, PhD

2 Maritime Scenario Maritime environment represents a challenging scenario for automatic video surveillance due to gradual and sudden illumination changes (e.g., clouds) motion changes (e.g., camera jitter) high frequency background objects (e.g., waves, raindrops) reflections Iniziativa Software 2 Page 2

3 System Architecture OBJECTIVE 3 DL KB SITUATION ASSESMENT + DL detection OBJECTIVE 1 IMAGE ANALYSIS tracking multiple camera data fusion feature extraction event understanding image classification validation radar + situation awareness AIS IR OBJECTIVE 2 DATA FUSION user OBJECTIVE 4 SITUATION AWARENESS

4 Objectives 1/4 1) Image Analysis Automatic analysis of the images coming from the cameras - Detection of the objects of interest - Tracking determines correspondences between a set of observations separated over time. Iniziativa Software 2 Page 4

5 Objectives 2/4 2) Data Fusion data coming from different sources - Video Day-Light - Video Infrared - RADAR - AIS are used to reduce the false positive detections Iniziativa Software 2 Page 5

6 Objectives 3/4 3) Situation Assessment a symbolic description of the observed scene is built - Feature based Image Classification Iniziativa Software 2 Page 6

7 Objectives 4/4 4) Situation Awareness An automatic reasoning system is able to respond appropriately as the situation evolves. - Track Split - Track Merge Iniziativa Software 2 Page 7

8 Detection Module OBJECTIVE 3 DL KB SITUATION ASSESMENT + DL detection OBJECTIVE 1 IMAGE ANALYSIS tracking multiple camera data fusion feature extraction event understanding image classification validation radar + situation awareness AIS IR OBJECTIVE 2 DATA FUSION user OBJECTIVE 4 SITUATION AWARENESS

9 Port Scenario A series of criticisms related to the port scenario have been individuated from the analysis of the literature: the use of Pan-Tilt-Zoom (PTZ) cameras, the presence of targets having very different size the presence of apparently motionless boats anchored off the coast. Iniziativa Software 2 Page 9

10 Background Subtraction VS Classification-based Detection Two different approaches Background Subtraction can be used on video sequences Classification-based Detection can be used with still images Iniziativa Software 2 Page 10

11 Foreground Extraction (Background Subtraction Technique) THRESHOLD T (based on illumination conditions) T foreground image current frame > background image blobs Iniziativa Software 2 Page 11

12 Background Subtraction Examples Waving Trees Water Surface Jug Page 12 12

13 Background Subtraction limitations Background subtraction is a common approach to detect moving objects in video sequences taken with a static camera. It is not straightforward to extend the approach to moving cameras. Cannot detect non-moving objects. Iniziativa Software 2 Page 13

14 Classification-based Detection current frame detection Iniziativa Software 2 Page 14

15 Pedestrian Detection Pedestrian detection in still images is considered among the hardest examples of object detection problems. Feature Extraction most informative object descriptors regarding the detection process are obtained from the visual content Detection the obtained object descriptors are utilized in a classification framework to detect the objects of interest. Iniziativa Software 2 Page 15

16 People Detection on Riemannian Manifolds Ensemble-of-feature approach (EOFA) (Tuzel et. al in PAMI 2008) A human is described as an ensemble of covariance matrices. Learning and feature selection is done simultaneously using LogitBoost of Riemannian Manifolds Iniziativa Software 2 Page 16

17 People detection: EOFA Results on Heathrow Zone A - Lobby Iniziativa Software 2 Page 17 17

18 Haar-like features POSITIVES NEGATIVES HAAR-LIKE FEATURES BASED CLASSIFICATION [Viola and Jones 2001, OpenCV] XML File Iniziativa Software 2 Page 18

19 Cascade of weak-classifiers Iniziativa Software 2 Page 19

20 Detection Scheme Input: Camera Frame, Haar Classifier Output: List of Observations (x,y) DETECTION XML file observations Workshop Iniziativa Software Page 20

21 Examples A LIVE test of the detector performance has been carried out at the VTS control center in Civitavecchia Iniziativa Software 2 Page 21

22 Sea-sky line vs Sea-coast line Since in presence of the coast the probability of finding false positives increases, erroneous detections laying above the seacoast line should be filtered out Sea-sky line Sea-coast line

23 Detection + Camera Heading Input: Camera Frame, Haar Classifier, Camera Heading Output: List of Observations, sea-sky line / sea-coast line Camera Heading DETECTION XML file sea-sky line observations Iniziativa Software 2 Page 23

24 Reflections and wakes Reflections and wakes on the water surface, can increase false positive detections Iniziativa Software 2 Page 24

25 Solution: Adding levels Iniziativa Software 2 Page 25

26 Detection Results Detection Rate False Alarm Rate Domenico Bloisi, Luca Iocchi, Michele Fiorini, Giovanni Graziano AUTOMATIC MARITIME SURVEILLANCE WITH VISUAL TARGET DETECTION Iniziativa Software 2 Page 26

27 Tracking Module DL KB OBJECTIVE 3 SITUATION ASSESMENT + DL detection OBJECTIVE 1 IMAGE ANALYSIS tracking multiple camera data fusion feature extraction event understanding image classification validation radar + situation awareness AIS IR OBJECTIVE 2 DATA FUSION user OBJECTIVE 4 SITUATION AWARENESS

28 Input: List of observations Visual Tracking Output: List of tracks (observation with an ID, filtered over the time) Tracking is crucial in filtering out false positives Iniziativa Software 2 Page 28

29 Validation Module OBJECTIVE 3 DL KB SITUATION ASSESMENT + DL detection OBJECTIVE 1 IMAGE ANALYSIS tracking multiple camera data fusion feature extraction event understanding image classification validation radar + situation awareness AIS IR OBJECTIVE 2 DATA FUSION user OBJECTIVE 4 SITUATION AWARENESS

30 VTS data VTS data are decoded in order to read boat information (name, System Track Number, latitude, longitude) and then projected on a Google Earth map Iniziativa Software 2 Page 30

31 Data Fusion First step: synchronization Time synchronization between VTS data and Video data Second Step: localization Localization of the VTS tracks navigating in the field of view of the camera Third Step: rotation VTS track rotation with respect to the camera heading Fourth Step: projection VTS tracks and Visual tracks are projected in a COMMON space Fifth Step: association Probabilistic association between VTS tracks and Visual Tracks Iniziativa Software 2 Page 31

32 VTS Track Transformation COMMON SPACE Iniziativa Software 2 Page 32

33 Visual Track Projection COMMON SPACE Iniziativa Software 2 Page 33

34 Data Association VISUAL TRACK VTS TRACK Iniziativa Software 2 Page 34

35 A Complete View DETECTION VISUAL TRACKING + VTS TRACKING SEA-SKY LINE EXTRACTION VTS DATA Iniziativa Software 2 Page 35

36 Multiple Radar Detections Visual analysis can help in correcting erroneous detection made by the radar. Detection results (a) are filtered over the time to obtain visual tracks (b) that are projected over a common space with radar and AIS tracks (c) Iniziativa Software 2 Page 36

37 Future Directions Iniziativa Software 2 Page 37

38 References Viola and Jones, Rapid object detection using a boosted cascade of simple features, CVPR 2001 Florian Adolf, How-to build a cascade of boosted classifiers based on Haar-like features Vadim Pisarevsky, OpenCV Object Detection: Theory and Practice Y. Freund, R.E. Schapire. A Decision-theoretic Generalization of On-line Learningand an Application to Boosting. Journal of Computer and System Sciences Iniziativa Software 2 Page 38

39 Automatic Maritime Surveillance with Visual Target Detection Domenico Bloisi, PhD