Video Processing for Surveillance and Security Appllications EuroSDR & ISPRS Workshop 2008 Geosensor Networks G. Saur M. Müller, Th. Müller, E. Monari, N. Heinze, W. Krüger, M. Esswein 1
Outline M3motion / Object Detection and Tracking Research Program AMROS Research Program NEST Experimental System ABUL 2
m3motion stable m 3 motion Stabil: Original Video Stabilized Video 3
m3motion mosaic m 3 motion Mosaic 4
m3motion detect m 3 motion Detect: Difference Image/Motion Detection Original Video/Vektor Field Of Camera Movement 5
motion detection IITB: Motion Detection in Moving Environment IITB: Human Tracking in Public Areas 6
tracking Color-based Target Tracking (Video) Color-Histogram-based Target Tracking (IR) 7
Research Program Autonomous Mobile Robots for Security Applications 8
Research Program AMROS AMROS is a Multisensoic system based on mobile robots for surveillance of endangered public and industrial objects (e.g. stadiums, waterworks, power plants, chemical facilities, etc.) operating in indoor and outdoor environments of real estates Features Autonomous patrolling to monitor security critical areas Easy to handle interactive user interface (Misson planning/ Teleoperation) Recognition of suspicious moving objects and changes in the sceneautonomous pursuit of persons and nearfield inspection (planned) 9
Research Program AMROS Visual Surveillance Navigation & Mapping 10
Research Program AMROS 2400 Mapping Simultaneous Localization and Mapping (SLAM) errors in the estimate of the robots pose correlated with the estimated map Based on scan-matching of LIDAR data and global optimization Very robust in clearly structured indoor environments Difficulties in unstructured outdoor environments Loop closure aided by absolute positioning 2200 2000 1800 1600 1400 1200 1000 800 500 1000 1500 2000 11
Research Program AMROS Multispectral Image Fusion 12
Research Program AMROS Human Detection 13
Research Program AMROS m3motion for Outdoor Surveillance 14
SOBCAH-Project Surveillance of harbours, & blue borders 15
Fields for SOBCAH-Project Information Exchange and Fusion Demonstration of Interoperability between Systems Automatic Sensor Orientation Control Triggered by Automatic Motion Detection Algorithms on Live Video Streams Triggered by Common Track Data Alarms (Offline Processing) Generation of additional Alarms (as Confirmation Signals) by Offline Processing Image Data Processing Video Based MTI Change Detection Tracking Spatial Image Fusion (Mosaicking) 16
SOBCAH-Project: Information Fusion Sobcah Shared Data (SSD) for Information Fusion Client-Server- Architecture SOBCAH Engine = Server ( IITB) SOBCAH C&C Tool + selected Exploitation Station(s) equipped with client (Selex S&AS, IITB) Subscription or query For specific data type For specific area Sensor Tasking by a communication tool Exploitation station gets alarm and contacts MI provider that observes the target area XMPP protocol (proposal) Chat Roster Management IQ message transfer (e.g. predefined by XML-schema) 17
SOBCAH-Project: Image Data Processing IITB Online Workstation IITB Offline Workstation 18
SOBCAH-Project: Image Data Processing IITB Workstation for Online Image Processing Image processing Stabilisation, video based MTI, change detection, tracking, etc. IP-Streams processing (M(J)PEG) of available video data Alarm generation (source-id, but without meta data) Results-Backup in SSD (if meta data available) IITB Workstation for Offline Image Processing Image processing for offline data Mosaiking, motion detection, etc. Common image data format including meta data needed for SDD access! Generation of additional alarms (as confirmation signals) Backup processing results in SSD in common format 19
Video Processing Setup (Online / Offline) Axis IP Server???? MSU/MCC SSD Client Offline Data Request Common Track Data for Alarm information! Net1 Net2 SSD Common Track Data for alarm information SSD Client Offline Data Request? Nemo CCTV IITB IP-CAM1 CCTV IITB IP-CAM2 IITB Image Processing Online Workstation IITB Image Processing Offline Workstation (incuding SSD Client) 20
Research Program Network Enabled Surveillance and Tracking 21
Research Program NEST NEST is a framework for distributed large collaborative sensor networks Generic approach Self-organizing sensor-clusters Expandability / Upgradebility NEST is designed to to be the next generation of network based surveillance system away from the camara-monitor oriented view, to the event and task oriented surveillance 22
Research Program NEST NEST: An object and task oriented surveillance system Video Camera Control Smart Cameras (Sensors + Detection Agents, SDAs) NEST NEST Services NEST Services Services Object Generation Video Processing Cluster NEST NEST Services NEST Services Services NEST Services (Tasks) NEST World Model (Abstract Description of the Situation/Environment) Real World 23
NEST System Overview Archiving Video HMI (Human Machine Interface) Plug & Protect NEST-Interface Mobile/Fixed Sensors Mobile/Fixed Actors Tracks Alarms Functional Components NEST Services External Services NEST Situation Model (Situation Awareness) Environment Model Observed Objects Web-based Situation Table Custom System Job Management Task Editor Task Control Task Manager 24
Experimental System ABUL Automatisierte Bildauswertung am Beispiel der Drohne UAV LUNA 25
ABUL M3motion Mosaicking Along track Stereo VIS / IR image processing Geocoding of video mosaics 26
Contact: Markus Müller, Dipl.-Inform. Manager Dept.»Autonomous Systems and Machine Vision«Tel.: +49 (0) 7 21 60 91 2 50 E-Mail: markus.mueller@iitb.fraunhofer.de Web: www.iitb.fraunhofer.de 27