Intelligent Camera Networks SECURING PUBLIC SPACES

Transcription

1 Intelligent Camera Networks SECURING PUBLIC SPACES OVERVIEW Airports and subway stations are equipped with numerous cameras in order to provide enhanced security for passengers in crowded locations. Intelligent vision technologies constitute an essential pillar for innovative safety and security solutions. Our research objective is to develop camera-network based large-scale visual awareness in order to enhance security in public spaces and infrastructures. To achieve this we focus on the following core topics: Camera platforms (hardware and embedded software) Video data management and exploration framework Networked multi-camera vision Multimedia information extraction Intelligent vision technologies help to accelerate border control while maintaining the highest level of security. AIT experts in this research field work in close collaboration with video component and security solution suppliers as well as with infrastructure operators and administrative authorities such as the Austrian Ministry of the Interior. Furthermore, there are close links to the work of other Departments within AIT, the Mobility Department in particular. Social aspects and privacy protection play a primary role to render these technologies acceptable for the society. For example, user surveys of public transportation systems suggest that the overwhelming majority of passengers are in favor of technologies such as video surveillance for enhanced security. However, it is essential that the public has clear information on how these systems and the resulting data are used.

2 CAMERA PLATFORMS HARDWARE AND EMBEDDED SOFTWARE Our goal is to bring important practical benefits to users of embedded Computer Vision in terms of advanced hardware design and embedded software for smart cameras and related devices such as video encoders. Our development strategy attempts to connect research with relevant product and user needs within the rapidly evolving domain of Computer Vision. We are in tight cooperation with Texas Instruments (TI) and benefit from our status as TI Early Adopter the only one in Europe, which provides us access to technical specifications 3-4 months in advance of their public release. Our strategic co-operation with Funkwerk video systeme (a major European camera manufacturer) and with a number of integrators and end-users provide us continuous feedback on technological developments and user requirements which are needed to accomplish vision-based solutions of great practical impact. We follow a three-fold development strategy: First, we realize hardware prototypes and Board Support Packages (BSP) as platforms for the development of various video processing applications (streaming, networked communication and interface implementations). Secondly, we use these prototypes as demonstration and validation platforms for our own video analytics concepts and technologies. Thirdly, we develop leading-edge technology (hardware and software) packages targeting specific vision tasks. Since hardware and software technologies are a rapidly expanding field, we are also following and adopting emerging trends such as mobile vision concepts based on ARM and Intel processors. APPLICATION EXAMPLE: Q7 MULTIMEDIA MODULE The development of the unique Q7 Multimedia Module marks a major milestone in our embedded systems development strategy. This module makes the main features of the TMS320DM8148 DaVinci Digital Media Processor available as a convenient and compact solution based on the well-established Qseven platform. This design includes the System-on-Chip and all of its high-speed core peripheral components mounted on a single piece of PCB, making TI s multimedia processing capabilities available to the user. An additional ISIF interface is given optionally (i.e. for connecting an optical sensor board), making the unit ready for camera applications.

3 VIDEO DATA MANAGEMENT INTERACTIVE EXPLORATION FRAMEWORK Our novel visual analytics framework is based on a service-oriented processing paradigm permitting optimal data, task and computational load distribution within a network of connected devices. Visual analysis, data handling and visualization are encapsulated within respective modules which are self-describing and work as scalable services on their own. Requirements of interconnectivity across various devices of heterogeneous character and scalability can be easily met: data are described in a light-weight data interchange format (JSON), while data handling, buffering and the underlying service framework are generated in an automated manner. APPLICATION EXAMPLE: INTERACTIVE VIDEO EXPLORATION Visual information in digital form is everywhere. Various authorities such as infrastructure operators and law enforcement agencies are confronted with search needs based on a visual description. Technological advances by steadily improving cameras, progress in Computer Vision, increasing computational power and the wide availability of network-based connectivity between devices are opening new horizons to tackle the task of finding the needle in a haystack of digital data. Several algorithmic components (e.g. detection, tracking, license plate recognition) have been combined and integrated into an interactive tool which provides search capabilities across various attributes and in joint feature spaces in large visual surveillance archives. This tool provides a convenient means to find specific video objects (such as a person or vehicle) in vast sets of video data. The system was awarded with the Best Demo Award from IBM at the 2011 IEEE Int. Conf. on Multimedia & Expo (ICME) in Barcelona.

4 NETWORKED MULTI-CAMERA VISION CAMERA CALIBRATION & LOCALIZATION By studying geometric properties of arbitrarily placed cameras and their spatio-temporal relation to moving people in a surveillance system, we have elaborated several possibilities of self-calibration. Using purely visual information we can fully calibrate a network of non- or slightly-overlapping cameras in an automated manner. Another patent-protected solution of great practical interest uses Structure from Motion (SfM) along with the support of an external image sequence acquired by a moving camera to infer the 3D poses of stationary surveillance cameras within a large-scale network. These auto-calibration and localization capabilities are especially relevant in large camera networks (airports, shopping malls, etc.). APPLICATION EXAMPLE: VISION-BASED LOCALIZATION Indoor localization is in general an unsolved problem, where satellite and radio based solutions fail or are too inaccurate. We have elaborated on an alternative passive solution which is based on the fact that environments exhibit unique visual appearance. This allows indoor localization to be cast as a purely visual problem. We have demonstrated that a surveillance camera or a mobile phone can be localized in real-time just by analyzing the content of its image and by matching it to a previously constructed visual map. The accuracy of the solution is far better than existing solutions. Image Location cloud computing 3D model

5 MULTIMEDIA INFORMATION EXTRACTION MAKING SENSE OF ABUNDANT DATA Automated means of reliably distilling raw visual input into high-level concepts such as objects, motion paths and object appearances are crucial analysis components of many vision systems (visual surveillance, media analysis). All stages of such a visual processing chain are associated with ambiguities, which imposes the primary scientific challenge when developing visual analytic functionalities. APPLICATION EXAMPLE: PEOPLE COUNTING IN A CROWD A Bayesian part-based pedestrian detection framework using shape and motion cues has been developed which relies on a maximum a posteriori (MAP) solution for human configurations in a crowd. The spatial configuration of detection hypotheses is optimized in a greedy manner w.r.t. the posterior probability and considering the occlusion status of individual hypotheses. Our patented-protected integral contour-based methodology provides an extensible framework, where variable shape templates for new object classes can be quickly integrated. The human detection framework is coupled with analytics functionalities quantifying the spatio-temporal human density, an information relevant in many application domains (retail, transportation). APPLICATION EXAMPLE: CONTENT FILTERING AND SEARCH Image content contained within the Internet encompasses vast amounts of visual data. Driven primarily by social responsibility internet providers and authorities strive to pinpoint data which can be gauged as critical in terms of visual content. Typical critical content images involve nudity and unwanted (e.g. right-extremist) propaganda. We have developed an automated recognition framework which employs a visual model of high representational power resulting in highly accurate filtering and search results when analyzing image data from the internet.

6 INTELLIGENT CAMERA NETWORKS REFERENCE PROJECTS INDUSTRIAL REFERENCE PROJECTS Funkwerk video systeme GmbH: HD H.264 surveillance camera and multichannel encoder PKE Videosysteme: Development and delivery of several thousand video encoding channels (PCI boards, encoders and software modules) for surveillance (e.g. Vienna Underground) Siemens AG: People re-identification and object detection algorithms TITAN Electronics: Embedded multichannel video encoder and object detection algorithms PROJECTS FUNDED BY THE AUSTRIAN RESEARCH PROMOTION AGENCY (FFG): Airport Security and Productivity (ASaP) Critical Content Filtering (CriFi) Future Border Control (FBC) Search for critical Events in Video Archives (SECRET) Passenger Monitoring in a large Network of Airport Surveillance Cameras (PAMON) PROJECTS FUNDED WITHIN THE EU FRAMEWORK PROGRAMME FP7: FastPass - a harmonized European border control system Engineering Proprioception in Computing Systems (EPiCS) Privacy by Design Surveillance Framework (PARIS)

7 INTELLIGENT CAMERA NETWORKS SECURING PUBLIC SPACES PROTOTYPES AND DEMOS MEP multimedia processing evaluation module: Q7 embedded multicore hardware (based on Texas Instrument DM8148) and software (board support package). Video Archive Search Manager 1.2: A highly configurable prototype that allows to search for objects, movements and similarities in video archives; Processing speed is 100 times faster than real time. Video Exploration Toolset: New generic concept and successor of Video Archive Search Manager; prototype of connected REST based analysis services for video archive investigations. Indoor Localisation: A demo on an Android TM mobile and a PC server where images are taken by the mobile and the server calculates the position and viewing direction of the user. Encoding and Streaming Software: Architecture and framework (LINUX) for video surveillance (based on Texas Instrument DM368). People detector: Software demo to detect people in AVI files or RTP video streams. Critical content detector: Software demo for detection of Swastika (or other) logos and nudity in images. Text Detector: Software Demo for finding text regions in images. AIT-OpenTLD Library: Robust visual tracking of objects in a video stream. AVC-1, AVC-4: Advanced Video Codecs: Embedded MPEG-4, H.264 encoding/decoding/streaming platforms (based on Texas Instrument DSPs DM642, DM648). CONTACT AIT Austrian Institute of Technology Safety & Security Department Donau-City-Straße 1, 1220 Vienna DI BERNHARD STROBL Intelligent Camera Networks Phone: +43(0) Fax: +43(0) bernhard.strobl@ait.ac.at Web: DR. MARKUS KOMMENDA Head of Video and Security Technology Phone: +43(0) Fax: +43(0) markus.kommenda@ait.ac.at Web: AIT Austrian Institute of Technology v Errors excepted. Images: AIT, istockphoto.com