Image processing in the military technology

Transcription

1 AARMS Vol. 2, No. 2 (2003) INFORMATICS ROBOTICS Image processing in the military technology TIBOR BUZÁSI Ministry of Defence, Technology Agency, Budapest, Hungary At the Ministry of Defence, Technology Agency (MoD TA) a number of algorithms have been developed to solve image-processing problems since Research has been based on the Cellular Neural Network Universal Machine (CNN UM) chip and its developer kit. Whereas the first year was devoted to the study of the CNN-UM chip, the years to follow were spent tracing the evolution of the CNN-UM. In 1997, several experiments were carried out with the chip, which at that time had 20x22 processing elements, binary input and output as well as optical input and an interface and developer software (VisMouse) developed by one of the institutes of the Hungarian Academy of Sciences. Initially, the CNN workgroup was composed of three specialists. What made the Institute interested in the CNN based technology is its special application for image processing and the fact that this robust and high-speed technological improvement has been promoted by an international expert group. It became clear over the years that the mathematical theorems described in the CNN paradigm could only be adapted to the VLSI technology, whereas the same procedure as integrated into other technologies proved to be rather difficult. As a consequence, although not all image-processing problems can be solved with the CNN technology, it is likely to make a revolution in the field of image processing. Since 1998 the year in which the work group was established the Institute has worked out morphological algorithms based on the CNN technology. The viability of the solution of other image-processing problems with the CNN was also considered. Since the middle of 1999 several motion detection and image fusion algorithms were created. Work continued on the chip that now has 64x64 processing elements, analogue input and output with a computational efficiency that is better than that of the earlier type. In the same year the aeroplane detection algorithm was successfully used with this brand new chip. The main objective of the work is to achieve the automatic evaluation of surveillance pictures. Received: November 10, 2003 Address for correspondence: TIBOR BUZÁSI Ministry of Defence, Technology Agency P.O. Box 26, H-1525 Budapest, Hungary tibor.buzasi@hmhti.hu

2 Introduction The present day computer technology is developing so fast that it is impossible to keep up with the newer and newer generations of processor and software families. The efficiency of PC doubles in every months. The question arises: is there a limit to this progress? There is a scale limit of the integrated circuits based on silicon, and nowadays we are approaching this limit. We are searching for new computational principles and their realisation. Nevertheless, digital technology has its own limits. Although the direction of the development is swinging to the real-time image processing applications, we can declare that this is the field where in some cases the digital technology is at its limits. CNN number of cells Table 1. The existing CNN universal machine characteristics München (1994) Sevilla (1995) Berkeley (1996) Helsinki (1997) Sevilla (1998) 12*12 20*22 16*16 48*48 64*64 Technology CMOS 2µ CMOS 0.8µ CMOS 1.2µ CMOS 0.5µ CMOS 0.5µ CNN model DTCNN DTCNN full range model CTCNN Chua-Yang CTCNN full range model CTCNN full range model Time constant 0.2 µs 250 ns 54ns 20ns 100ns Input analogue binary analogue binary analogue Output Equivalent binary binary analogue/ binary binary binary processing 68 Gop/s 180 Gop/s 655 Gop/s 11.7 Top/s Top/s efficiency/cm 2 In the beginning of the 80-s Dr. Tamás Roska (Hungarian Academy of Sciences, Computer and Automation Institute) and L. O. Chua (Berkeley University) elaborated a processor with a new architecture. The theory is based on Nature: the CNN processor models the natural nervous system, that is, the CNN processor is analogous with the natural nervous system in architecture and behaviour, this is what the name CNN refers to. In contrast with today s digital technique, the CNN s processing is mainly analogue. The interactions depend on the connectivity of the neighbouring cells. Similarly, the 222 AARMS 2(2) (2003)

3 interactions depend on the sphere of the CNN cells and their connectivity. We can easily imagine a CNN computer as a set of many elementary analogue processing elements. The processing in these elementary computers is done parallel. The advantage of this technology is manifested in the field of image processing, where the whole image is processed parallel (in contrast with the presently applied digital technique, where the data processing is serial). Since the CNN technology has not been popular, its development is behind that of the digital, but its price exceeds it. The processing speed of the various CNN chips can be seen in Table 1. For the sake of comparison let us say that there is at least one task that a 20*22 chip which was produced in 1995 can process faster than a personal computer based on a PII 300 processor. Correspondingly, the 64*64 chip which was produced in 1998 is faster by order of magnitude. The application of CNN in military technology One disadvantage of aerial reconnaissance is that only the analysing method of the aerial pictures is not automated. As a result, the information can be evaluated some hours after pictures were taken. This is a serious disadvantage in both military and civil applications. One solution of this problem is that an operator (analyser) observes the display continuously. This however can not always be done since the reconnaissance is carried out on board of the UAV and in certain cases the transmission of the aerial picture to the ground is impossible in real time. This problem can be solved with the aid of a device, which can be programmed to detect a previously defined shape. When this automated system finds the object, the camera can be focused on it or it can transmit the co-ordinates of the object or start any other predefined operation. The solution of the problem demands a lot of computing time, because PAL video signal must be processed in real time, which means we want to recognise the targets and detect the motion simultaneously. A possible way of solving the problem is the application of CNN. It provides the necessary artificial sight. The targets and their morphological features must be defined, so that the appropriate program can separate them from the background. The MOD s IMT Automation and Radar Department have implemented the task in co-operation with the Computer and Automation Institute of the Hungarian Academy of Sciences. Studies were made on object recognition and motion detection. Before the exposition of the developed algorithms we outline the working principle of the CNN processor. AARMS 2(2) (2003) 223

4 Introduction to the theory of CNN The Cellular Neural Network (CNN) is an analogue and dynamic processor array, in which the processing elements are arranged in a regular geometrical structure, the elements form a connected network. 1 After the initialising step through temporary states the system stops in its stable state. To return to our comparison with the nature we can imagine the cells as the cerebral neurones with dynamical synopsis. The processing elements are placed in the nodes of a grid and they have connection with the neighbouring elements. Neighbour cells are those that are placed nearer than r. The C(i,j) cell s neighbours are: N(Ci,j)={C(k,l) k-i ~ r, l-j ~ r}. In the simplest CNN (r = 1) the processing elements neighbours are the eight closest elements. The state equation* The state equation is as follows: x& i j xi, j + c N( c ) Ai, j; k, l yk, l + c N( c ) Bi, j; k, luk, l + zi,, (Eq. 1), k, l i, j k, l i, j j where x, y, u are the cell state, output, input, A, B matrixes are showing the effect of the neighbouring cells, the Z matrix is a simple threshold function (it can be different for each cell). CNN with space invariant template Figure 1. CNN with space invariant template * The next two equations show that the CNN chip can be implemented easily and it can be a fast analogue computational non-linear processor. 224 AARMS 2(2) (2003)

5 In this case the A, B, Z, matrixes (they are together called template) are space invariant (each cell has the same A, B, and Z matrix). In this case when r = 1 the behaviour of the network can be described with the A, B, Z matrixes 19 parameters. We can get a better problem solver CNN if the A, B and Z matrixes are functions instead of constant parameters. 1 Output function 1 1 y i, j = f ( xi, j ) = xi, j + 1 xi, j 1 (Eq. 2) 2 2 Figure 2. Diagram of the output function The CNN processors manufactured so far are linear, where the A, B and Z matrixes can not be defined as functions. In the light of the above description one can easily find out that the programming of a CNN processor differs from the programming a traditional processor. We can get the result by simplifying the programming procedure that is by placing the image in the input of the processor and by using templates. AARMS 2(2) (2003) 225

6 Results Plane detection What you can see in the Figure 3 is a picture from an original aerial film showing two aeroplanes in an airfield. 2 The software developed for CNN evaluates the picture with three different filtering methods in the course of which the software processes the contents of the picture in 3 phases. First it performs cross detection, then contrast threshold and finally finds local minimum. This method, of course, is advantageous for the detection of planes only; the identification of a tank requires a different method. Figure 3. Detected planes The software was able to identify the two planes. It is capable of the same performance when identifying aeroplanes in a moving image. 226 AARMS 2(2) (2003)

7 Sensor fusion Figure 4. Normal (left) and thermo (right) photos of an object Figure 5. Fusion photos of an object A different technique of image processing is sensor fusion. In this case, daylight and infrared photos of the same objects (Figure 4) are processed. 2 The great advantage of this technique is the highlighting of important details. During the first fusion (Figure 5, left), the two pictures were simply integrated into each other. In the second fusion (Figure 5, right) the picture of the thermo camera is complemented by the daylight camera in such a way that important details (in this case, two men) become more prominent. AARMS 2(2) (2003) 227

8 Object extraction In certain cases, when we know which part of the picture we want to observe, we can use the method of segmenting. We are now looking for moving objects over the horizon line (Figure 6). Figure 6. Flying object The horizon line is generated dynamically from picture to picture. The observed object is in the segment above the horizon line, indicated with a red circle in the program. As the upper segment is homogeneous, the likelihood of error is minimal. Accuracy can be further increased if a morphological description is available of the searched object. Motion detection algorithms The characteristics of CNN make it possible to distinguish static objects from moving objects. These two men in the infrared photo can be filtered out as moving objects (Figure 7). 228 AARMS 2(2) (2003)

9 Figure 7. Photo from thermo camera image sequence Figure 8. Static background and detected moving objects The software is capable of separating the static background from the moving objects. Furthermore this software is able to display moving objects without their background (Figure 8). This application can be very effective in evaluating air photos when searching moving objects. AARMS 2(2) (2003) 229

10 Real-time map making* There are several cases when we need a real time aerial photo about the terrain. In these cases we need only a UAV, which can fly above the area. From the video sequence we can generate a dynamic picture. In this picture we can find all the objects of the terrain and easier to define different points of locations. First the track line of the flying helicopter is detected. On the base of the previously computed track line information the territory map is reconstructed (Figure 9). Figure 9. Rotated map computed from video image sequence Summary The most important objective of reconnaissance is to provide pictures asap irrespective of weather conditions. The latest developments of computer technology in both the hardware and the software fields make the evaluation of pictures possible in a relatively short time. In the course of the Iraqi and Kosovo air campaigns commanders demanded pictures of the battlefield situation continuously in order to be able to assess the position of own forces and enemy forces and the state of their military equipment. Further steps to be taken were determined on the basis of this assessment. The most effective imaging devices are daylight and IR cameras, daylight and infrared sensors as well as synthetic aperture radar operating on several frequency bands. These devices have considerably improved, but the information they provide can be misleading. The most reliable method seems to be sensor fusion, in which data arriving from a given number of sensors is channelled into the evaluating system. The fusion method gives unambiguous information about a target or a military object. The method is becoming increasingly relevant as a consequence of the widespread use of concealment, camouflaging and * A practical scanning device was made in 2000 at VTULaPVO PRAHA institute applicable for air reconnaissance, but it does not have the capability of image distortion reconstruction. In the year of 1995 a French made thermo scanner was adjusted to the Sojka type UAV. 230 AARMS 2(2) (2003)

11 deception in the last few years. The task of reconnaissance is complicated by a large degree of mobility, limiting the validity of the assessment to a certain moment in time. This means that the time available for the evaluation of data, reporting and the implementation of countermeasures is critically short. Only if all of them can be completed within such a short period of time there is a possibility for accurate target identification, weapon direction and destruction. Otherwise the mission fails. References 1. T. BUZÁSI, B. SZÉKELY: The application of the cellular neural network (cnn) in military technology, In: Proceedings of the RCMCIS 99 (Zegrze, Poland, Oct. 6 8) pp T. BUZÁSI, B. SZÉKELY, I. NAGY, L. GÉRESI, Z. VARGA: The application of the cellular neural network (CNN) in military technology. Regional Conference on Military Communication and Information Systems 2000, October 4 th 6 th, Zegrze, Poland. 3. L. O. CHUA, T. ROSKA, T. KOZEK, Á. ZARÁNDY, CNN universal chips crank up the computing power, IEEE Circuits and Devices, (1996) L. O. CHUA, L. YANG, Cellular neural networks: Theory; Cellular neural networks: Applications, IEEE Transactions on Circuits and Systems, 35 (1998) R. DOMINGUEZ-CASTRO, S. ESPEJO, A. RODRIGUEZ-VAZQUEZ, R. CAMONA, A CNN universal chip in CMOS technology, Proc. of the Third IEEE Int. Workshop on Cellular Neural Networks and their Applications (CNNA-94), Rome, 1994, pp T. ROSKA, L. O. CHUA, The CNN universal machine: An analogic array computer, IEEE Transactions on Circuits and Systems, II: Analog and Digital Signal Processing, Special Issue on CNN, 40 (1993) K. LOTZ: A látó- és hallórendszer néhány részmodelljének implementálása és alkalmazása celluláris neurális hálózat típusú processzortömbökkel, Kandidátusi értekezés, MTA SZTAKI, T ROSKA, C. BECCARI, M. BIEZ, P. P. CIVELLERI, M. GILLI, Design Automation Day on Cellular Visual Microprocessor, Stresa-Italy, August J. CANNY: A computational approach to edge detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8 (1986) J. S. LIM: Two-Dimensional Signal and Image Processing. Englewood Cliffs, NJ: Prentice Hall, 1990, pp J. R. PARKER: Algorithms for Image Processing and Computer Vision. New York: John Wiley & Sons, Inc., 1997, pp R. C. GONZALEZ, R. E. WOODS: Digital Image Processing. Addison-Wesley, 1992, p R. M. HARALICK, L. G. SHAPIRO: Computer and Robot Vision, Volume I. Addison-Wesley, 1992, p AARMS 2(2) (2003) 231