Studies in Computational Intelligence Volume 481 Series Editor J. Kacprzyk, Warsaw, Poland For further volumes: http://www.springer.com/series/7092
Aleksander Nawrat Zygmunt Kuś Editors Vision Based Systems for UAV Applications ABC
Editors Prof. Aleksander Nawrat Silesian University of Technology Gliwice Poland Dr. Zygmunt Kuś Silesian University of Technology Gliwice Poland ISSN 1860-949X ISSN 1860-9503 (electronic) ISBN 978-3-319-00368-9 ISBN 978-3-319-00369-6 (ebook) DOI 10.1007/978-3-319-00369-6 Springer Cham Heidelberg New York Dordrecht London Library of Congress Control Number: 2013934960 c Springer International Publishing Switzerland 2013 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
In God We Trust, All others we observe andkeepveryclose... To our friends
Preface The book was motivated by a significant number of vision based algorithms for Unmanned Aerial Vehicles (UAV) that are developed during research and development projects. Nowadays, it can be observed a significant increase in a number of UAV applications. One of the main factors in the development of UAV is rapid progress of development of advanced sensory and considerable increase of computing power. Aerial observation is currently used for both military and civilian purposes. The variety of drones and their parameters gives almost unlimited possibilities in areas like vision surveillance, cartography, geodesy, cinematography or meteorology. Most if not all of these applications utilize some kind of vision based sensor like video camera, thermal imaging camera or laser range finders. Therefore it is essential to improve the existing general video acquisition devices and design and construct innovative dedicated for special purposes cameras. The amount of high and low frequencies disruptions is significantly greater than for traditional handheld cameras. It is a matter of high importance to design devices to reduce the impact in order to deliver proper video stream stabilization. Additionally bearing capacity of quick response class UAV is highly limited. Therefore video acquisition devices are required to be as light as possible but at the same time rugged and reliable. However, there is also a considerable group of vision based control algorithms that are used e.g. for target detection or collision-free navigation. It is worth mentioning about the task of controlling a group of mobile robots within line of sight. There is also an active field of classification of actions and patterns based on information often fused from multiple video stream sources. Finally the video stream has to be compressed and transmitted wirelessly from UAV to ground station over heterogeneous networks with different dynamically changing topologies. Main idea of visual surveillance is to detect any specified targets or abnormal situations in the line of sight. Targets can be detected due to their characteristic features e.g. face, character of movement, size, color or shape. Those features are detected, tracked and recognized by a dedicated set of algorithms processing images acquired from live video stream. Detection and recognition are usually a time consuming thus there is object tracking phase in the middle. The main challenge of such algorithms is to reconcile the tracking quality with simplicity required for real time processing. Regardless the application object recognition is an essential task for complex systems where a certain amount of system autonomy is required. The book contains
VIII Preface valuable information in the field of detection, tracking and recognition of objects. Presented challenges and solutions are the results of research conducted by the team. Nowadays, image acquisition devices are mainly used by human operators or large tactical or strategic class UAVs. Therefore there is a need for innovative video sources that are characterized by small weight, low energy consumption and at the same time high resistance to vibrations, shock and temperature. Traditionally used visible light cameras cover only a small part of the magnetic spectrum and are easily covered by dust, smoke, fog, etc. In order to continue aerial observation thermal imaging and omnidirectional cameras have to be used. It is commonly accepted that over 80% of all perceptual information being received by human brain is perceived through eyes. It is natural for control algorithms developers to try to utilize the information as it is done by both human and animal brains. Vision information can be utilized not only for control a single device but also for controlling a group of e.g. mobile robots. An alternative approach is to use aerial vision information for controlling detected and recognized objects on the ground. Main aim of design and construction of UAVs is their practical application. However, their design is a lengthy process and requires intensive evaluation using both simulation and experimental data. Even then, it is often necessary to apply amendments based on the actual real life experience of the application. Misrecognition can sometimes lead to irreversible consequences therefore in order to improve the quality of recognition frequently fusion of data from multiple sources of information is used. However it is worth mentioning that before the video stream from a camera mounted on an UAV it is necessary to wirelessly transmit to human operator using ground base station. In addition, video data transmission requires a significant network bandwidth. Communication over long distances is charged a considerable limitation in this regard. Therefore it is necessary to use a dedicated image and sound compression algorithms. The topics covered in this book also includes practical applications of classification algorithms used in video streams for UAV. This book is not only intended for researchers involved in image processing field. Aim of the book was to create a valuable source of information for researchers and constructors of solutions utilizing vision from UAV. Scientists and researchers involved in computer vision, image processing, data fusion, control algorithms, mechanics, data mining, navigation and IC can find many valuable, useful and practical suggestions and solutions. The latest challenges for vision based systems are also presented. Furthermore the information included in the book can benefit greatly students of higher years of study in the fields of computer science, automation, robotics, mechanical engineering or data mining. The author strongly encourage all readers to explore the following chapters of the book and invite to open and honest discussion of any questions or concerns encountered during the lecture. Gliwice February 2012 Aleksander Nawrat Zygmunt Kuś
Acknowledgments This work has been supported by Applied Research Programme of the National Centre for Research and Development as a project ID 178438 path A - Costume for acquisition of human movement based on IMU sensors with collection, visualization and data analysis software.
Contents Part I: Design of Object Detection, Recognition and Tracking Algorithms Machine Vision in Autonomous Systems of Detection and Location of Objects in Digital Images... 3 Artur Babiarz, Robert Bieda, Karol Jędrasiak, Aleksander Nawrat Recognition and Location of Objects in the Visual Field of a UAV Vision System... 27 Robert Bieda, Krzysztof Jaskot, Karol Jędrasiak, Aleksander Nawrat Automatic Targeting Sentry Turret for Distributed Systems... 47 Piotr Demski, Tomasz Grzejszczak, Karol Jędrasiak, Michał Mikulski Object Tracking for Rapid Camera Movements in 3D Space... 57 Zygmunt Kuś, Aleksander Nawrat Object Tracking in a Picture during Rapid Camera Movements... 77 Zygmunt Kuś, Aleksander Nawrat Part II: Construction of Image Acquisition Devices Used in UAV Applications Thermal Camera for Autonomous Mobile Platforms... 95 Grzegorz Bieszczad, Michał Krupiński, Henryk Madura, Tomasz Sosnowski Gyro-Stabilized Platform for Multispectral Image Acquisition... 115 Damian Bereska, Karol Jędrasiak, Krzysztof Daniec, Aleksander Nawrat Omnidirectional Video Acquisition Device (OVAD)... 123 Sławomir Fraś, Karol Jędrasiak, Jan Kwiatkowski, Aleksander Nawrat, Dawid Sobel
XII Contents Part III: Design of Vision Based Control Algorithms Vision System for Group of Mobile Robots... 139 Artur Babiarz, Robert Bieda, Krzysztof Jaskot A Distributed Control Group of Mobile Robots in a Limited Area with a Vision System... 157 Artur Babiarz, Robert Bieda, Krzysztof Jaskot System for Multi-axial Mechanical Stabilization of Digital Camera... 177 Damian Bereska, Krzysztof Daniec, Sławomir Fraś, Karol Jędrasiak, Mateusz Malinowski, Aleksander Nawrat Probabilistic Approach to Planning Collision Free Path of UAV... 191 Dawid Cedrych, Adam Gałuszka, Marcin Pacholczyk, Krzysztof Skrzypczyk, Aleksander Nawrat Part IV: Practical Applications of Classification Algorithms Used in Video Streams for UAV Information Fusion in Multi-agent System Based on Reliability Criterion... 207 Martin Mellado, Krzysztof Skrzypczyk Prototyping the Autonomous Flight Algorithms Using the Prepar3D R Simulator... 219 Krzysztof Daniec, Paweł Iwaneczko, Karol Jędrasiak, Aleksander Nawrat Feature Extraction and HMM-Based Classification of Gait Video Sequences for the Purpose of Human Identification... 233 Henryk Josiński, Daniel Kostrzewa, Agnieszka Michalczuk, Adam Świtoński, Konrad Wojciechowski The Dedicated Wireless Communication Device for Group of Unmanned Vehicles... 247 Krzysztof Daniec, Karol Jędrasiak, Roman Koteras, Aleksander Nawrat, Tadeusz Topór-Kamiński Selection of Individual Gait Features Extracted by MPCA Applied to Video Recordings Data... 257 Henryk Josiński, Agnieszka Michalczuk, Andrzej Polański, Adam Świtoński, Konrad Wojciechowski Comparative Analysis of Power Loses in Selected Measurement Devices Used in SMART GRID Systems... 273 Aleksander Nawrat, Anna Piaskowy, Artur Skórkowski, Tadeusz Topór-Kamiński
Contents XIII Technology Development of Military Applications of Unmanned Ground Vehicles... 293 Tomasz Czapla, Józef Wrona Estimation of Solution of Discrete Linear Time-Varying System... 311 Adam Czornik, Aleksander Nawrat, Michał Niezabitowski Virtual Reality Technology for Military and Industry Training Programs... 327 Marek Koźlak, Antoni Kurzeja, Aleksander Nawrat Verifying Unmanned Land Vehicle Vision System Configuration Using Serious Gaming Solution... 335 Marek Koźlak, Antoni Kurzeja, Aleksander Nawrat Conclusions... 343 Author Index... 345
List of Contributors Artur Babiarz e-mail: artur.babiarz@polsl.pl Damian Bereska e-mail: damian.bereska@polsl.pl Robert Bieda e-mail: robert.bieda@polsl.pl Grzegorz Bieszczad Military University of Technology, Institute of Optoelectronics, Warsaw, Poland e-mail: gbieszczad@wat.edu.pl Dawid Cedrych e-mail: dawid.cedrych@polsl.pl Tomasz Czapla Silesian University of Technology, Faculty of Mechanical Engineering, Division of Theoretical and Applied Mechanics, ul. Konarskiego 18A, e-mail: tomasz.czapla@polsl.pl
XVI List of Contributors Adam Czornik e-mail: adam.czornik@polsl.pl Krzysztof Daniec e-mail: krzysztof.daniec@polsl.pl Piotr Demski e-mail: piotr.demski@polsl.pl Sławomir Fraś e-mail: slawomir.fras@polsl.pl Adam Gałuszka e-mail: adam.galuszka@polsl.pl Tomasz Grzejszczak e-mail: tomasz.grzejszczak@polsl.pl Paweł Iwaneczko e-mail: pawel.iwaneczko@polsl.pl Krzysztof Jaskot e-mail: krzysztof.jaskot@polsl.pl Karol Jędrasiak e-mail: karol.jedrasiak@polsl.pl
List of Contributors XVII Henryk Josiński Silesian University of Technology, Institute of Computer Science, Akademicka 16, e-mail: henryk.josinski@polsl.pl Daniel Kostrzewa Silesian University of Technology, Institute of Computer Science, Akademicka 16, e-mail: Daniel.kostrzewa@polsl.pl Roman Koteras e-mail: roman.koteras@polsl.pl Marek Koźlak Ośrodek Badawczo-Rozwojowy Urządzeń Mechanicznych OBRUM sp. z o.o. ul. Toszecka 102, 44-117 Gliwice, Poland email: mkozlak@obrum.gliwice.pl Silesian University of Technology, Technopark, ul. Konarskiego 18C, 44-100 Gliwice, Poland email: marek.kozlak@polsl.pl Michał Krupiński Military University of Technology, Institute of Optoelectronics, Warsaw, Poland e-mail: mkrupinski@wat.edu.pl Antoni Kurzeja Ośrodek Badawczo-Rozwojowy Urządzeń Mechanicznych OBRUM sp. z o.o. ul. Toszecka 102, 44-117 Gliwice, Poland email: akurzeja@obrum.gliwice.pl Zygmunt Kuś e-mail: zygmunt.kus@polsl.pl Jan Kwiatkowski e-mail: jan.kwiatkowski@polsl.pl
XVIII List of Contributors Henryk Madura Military University of Technology, Institute of Optoelectronics, Warsaw, Poland e-mail: hmadura@wat.edu.pl Mateusz Malinowski e-mail: mateusz.malinowski@polsl.pl Martin Mellado Instituto de Automatica e Informatica Industrial, Universidad Plitecnica de Valencia e-mail: marin@ai2.upv.es Agnieszka Michalczuk Silesian University of Technology, Institute of Computer Science, Akademicka 16, e-mail: agnieszka.michalczuk@polsl.pl Michał Mikulski e-mail: michal.mikulski@polsl.pl Aleksander Nawrat e-mail: aleksander.nawrat@polsl.pl Ośrodek Badawczo-Rozwojowy Urządzeń Mechanicznych OBRUM sp. z o.o. ul. Toszecka 102, 44-117 Gliwice, Poland email: anawrat@obrum.gliwice.pl Michał Niezabitowski e-mail: michal.niezabitowski@polsl.pl Marcin Pacholczyk e-mail: marcin.pacholczyk@polsl.pl
List of Contributors XIX Anna Piaskowy Silesian University of Technology, e-mail: anna.piaskowy@polsl.pl Andrzej Polański Silesian University of Technology, Institute of Computer Science, Akademicka 16, e-mail: andrzej.polanski@polsl.pl Artur Skórkowski Silesian University of Technology, e-mail: artur.skorkowski@polsl.pl Krzysztof Skrzypczyk e-mail: Krzysztof.skrzypczyk@polsl.pl Dawid Sobel e-mail: dawid.sobel@polsl.pl Tomasz Sosnowski Military University of Technology, Institute of Optoelectronics, Warsaw, Poland e-mail: tsosnowski@wat.edu.pl Adam Świtoński Silesian University of Technology, Institute of Computer Science, Akademicka 16, e-mail: adam.switonski@polsl.pl Tadeusz Topór-Kamiński e-mail: tadeusz.kaminski@polsl.pl Konrad Wojciechowski Silesian University of Technology, Institute of Computer Science, Akademicka 16, e-mail: konrad.wojciechowski@polsl.pl
XX List of Contributors Józef Wrona Military University of Technology, ul. gen. Sylwestra Kaliskiego 2, Warsaw, Poland, Industrial Research Institute of Automation and Measurements, Aleje Jerozolimskie 2022, 02-486 Warsaw, Poland e-mail: jwrona@wat.edu.pl