An Energy-Based Vehicle Tracking System using Principal Component Analysis and Unsupervised ART Network

Transcription

1 Proceedings of the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES (AIKED '9) ISSN: ISBN: An Energy-Based Vehicle Tracking System using Principal Component Analysis and Unsupervised ART Network A. SRIKAEW, P. KUMSAWAT, K. ATTAKITMONGCOL, N. SROISUWAN AND C. SOTTHITHAWORN Robotics & Automation Research Unit for Real-World Applications School of Electrical Engineering, Suranaree University of Technology 111 University Ave., Nakhon Ratchasima, THAILAND santa electrical Abstract: This work presents an automatic vehicle detecting and tracking system from a sequence of images. The vehicle detection system uses energy-based images including symmetry energy, Gabor energy, and road energy, to initially locate vehicles in each image. The tracking system then utilizes the adaptive resonance theory network for vehicle recognition and tracking based on vehicle energy images. The vehicle energy images are fed into the network which can automatically recognize salient features of vehicles by analyzing theirs principal components. This unsupervised network allows the system to efficiently perform tracking in dynamic environments where shapes and sizes of vehicles are changing all the time. By using the vehicle energy model, the proposed system can also track multiple vehicles simultaneously, both frontal and rear view. Results and discussions are described. Key Words: Vehicle detection, Vehicle tracking, Symmetry energy, Gabor energy, Road energy, Principal component analysis, Adaptive resonance theory network 1 Introduction Nowadays, there are many kinds of technology for safety and reducing accidents on the road. Statistically, most of accidents come from vehicle driver. Driver assistance system then has been focused to warn the driver for a chance of an accident to be happening or to control vehicle for avoiding any wreck. Generally, various kinds of sensors can be installed on a vehicle to detect surrounding objects which mostly are other vehicles or pedestrians. There are two types of these sensors: active sensor and passive sensor. Active sensors, such as lidar, radar, and laser, are capable of scanning areas around the vehicles, but they have low resolution, slow scan speed, and expensive. Passive sensors, such as camera, are simple to use, less expensive, high resolution, and useful for other simultaneous tasks, e.g. lane detection and traffic sign recognition. Researches of vehicle tracking can be categorized into three groups [1]: optical flow based, model based and feature based vehicle tracking. The optical flow based vehicle tracking utilizes useful information from optical flows. This approach is efficient but required to perform with high resolution images. The model based vehicle tracking is to estimate model in the image plane and use it for matching and tracking. The feature based vehicle tracking deploys salient features of vehicle such as edges, textures, and corners, for matching vehicle in the image. In [2], information of HSV is applied for tracking vehicle using both hue level and edge image for building statistic model of vehicle. In [3], feature vectors of vehicles are constructed using areas and positions of rectangle in the image, and averaging color of vehicle. The Kalman filter is used to predict position of vehicle in the next frame. In [4], Kanade-Lucas-Tomasi (KLT) feature tracker is exploited with Kohonen self-organizing map (SOM) network for tracking vehicles. Input of the network consists of both x and y component of vehicle velocity. Mainly, the vehicle tracking system is composed of two systems: vehicle detection and vehicle tracking. The vehicle detection system is responsible for finding vehicles within the images and then the vehicle tracking system can identify and track each vehicle in the image continuously. Vehicle tracking is a task that keeps track of road conditions. This allows useful data for applications of reducing accidents on the road. Performance of vehicle tracking system depends on the ability of vehicle detection. This work has been extended from [5] in which the system utilizes vehicle features from principal component analysis of the Gabor vehicle images. These vehicle features are examined by the adaptive resonance theory network to recognize and keep track of each vehicle separately without any supervised training. The system is then suitable for

2 Proceedings of the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES (AIKED '9) ISSN: ISBN: Figure 1: Vehicle detection subsystem Figure 3: Symmetrical characteristic of vehicle in different bounding boxes Figure 2: Vehicle bounding boxes from different distances tracking vehicle in dynamic environment where the size and view of vehicle are changing in every frame. 2 Vehicle Tracking System The proposed vehicle tracking system is mainly composed of two subsystems: vehicle detection and vehicle tracking. The vehicle detection subsystem can automatically locate any vehicle within the image by using energy-based models. The vehicle tracking subsystem utilizes adaptive resonance theory network to track vehicle without any supervised learning. Principal component analysis of vehicle image is deployed as the network input. Details of both subsystems are presented in the following subsections. 2.1 Vehicle Detection Subsystem A diagram of the vehicle detection subsystem is shown in Fig. 1. From the diagram, vehicles in the image are searched and located using perspective relationship between vehicle size and distance from the camera [6]. The results from this technique can reduce time of computation, also the area of interest. Fig. 2 demonstrates vehicle bounding boxes in different distances. The resulting bounding boxes are then examined to detect any presence of vehicle. The presence of vehicle is detected using vehicle energy. The vehicle energy consists of symmetry energy, Gabor energy, and road energy. The symmetry energy is computed based on the vehicle appearance in the image. The edge image is calculated using Sobel operator in horizontal direction. The accumulator array of edge pixels are used to find symmetrical axis. This symmetrical axis allows the system to eliminate any bounding box that is not well positioned for the vehicle. Examples of such characteristic are depicted in Fig. 3. The vehicle symmetry energy (E s ) is then computed by E s = α max A (1) where A is the accumulator array of edge pixels and α is symmetry correction factor. The vehicle Gabor energy utilizes Gabor filter to extract vehicle features using equation (2). By choosing appropriate w, the resulting Gabor filtered image can enhance vehicle appearance and suppress any irrelevant information such as road or background (see Fig. 4). G(w) = exp log (w/w o) 2 2 log (k/w o ) 2 (2) Vehicle Gabor energy can be computed from amplitude of Gabor filtered image using equation (3) (see vehicle model in Fig. 5) where An 1A, An AB, and An BC are Gabor amplitude of row 1 to A, A to B, and B to C, respectively. E G = max An 1A + max An AB + max An BC (3) The road energy is determined by using saturation (S) component of HSV color model of the image. A

3 Proceedings of the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES (AIKED '9) ISSN: ISBN: Figure 4: Vehicle response in Gabor filtered image Figure 6: Saturation component of road images Figure 5: Vehicle Gabor model sufficient low value of S indicates area of the road within the image as can be seen in Fig. 6. The road energy can be computed from summation of saturation value of pixels within an area below the vehicle bounding box using equation (4) where R s is road saturation value inside the bounding box, m is number of rows, n is number of columns and β is the road correction factor. Fig. 7 demonstrates the road bounding box which locates below the vehicle bounding box. m n E r = β R s (i, j) /(m n) (4) i=1 j=1 Figure 7: Road bounding box The vehicle symmetry energy, Gabor energy, and road energy are combined to obtain vehicle enery using the following equation. E v = (E s + E G ) (E r + ɛ 1 + ɛ 2 ) (5) The value of ɛ 1 and ɛ 2 are correction factor for oversize and undersize vehicle bounding box, respectively. The greater vehicle energy within the image, the greater opportunity to find vehicle at that location. The image is searched from the vanishing line to locate the positions having vehicle energy above the predefined value. An example of vehicle energy is shown in Fig. 8. The resulting vehicle images are then used as input for vehicle tracking subsystem as described in the following section. Figure 8: Vehicle energy example

4 Proceedings of the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES (AIKED '9) ISSN: ISBN: (a) (b) Figure 9: Vehicle tracking subsystem 2.2 Vehicle Tracking Subsystem The main objective of the vehicle tracking subsystem is to continuously locate each vehicle in the sequence of images. The problem rises when the vehicles are moving because they appear differently in sizes and views at different time frame. Consequently, the system must be capable of dynamically tracking of vehicles. In this work, the vehicle tracking subsystem consists of two parts: vehicle feature extraction and vehicle tracking (see diagram in Fig. 9). For vehicle feature extraction, a vehicle grayscale image is filtered by log-gabor filter. Next, the principal component analysis (PCA) of the filtered image is computed. In additions, the PCA is also calculated from a vehicle RGB image. Results of both PCAs are combined which is called vehicle feature vector. This vector is then examined by the adaptive resonance theory network for vehicle tracking. The vehicle feature vector has substantially smaller size of the vehicle bounding box image. The same vehicle in consecutive images provides very similar vehicle feature vector as can be seen in Fig. 1. Finally, these vehicle feature vectors are then recognized by adaptive resonance theory (ART) network [7] to identify and track each vehicle frame by frame. The vehicle feature vector is created and stored in the weight layer of the ART network. The new vector from the next frame is compared to check for the similarity with the weights in the network. If it is suf- (c) Figure 1: (a) Vehicle feature vector of vehicle #1 (b) vehicle feature vector of different vehicle (#2) (c) vehicle feature vector of same vehicle in consecutive image frame ficiently similar to one of the weight in the memory, it can be recognized as the same vehicle from the previous frame. The corresponding weight is then updated. This implies that the size and view of vehicle are also updated. The number of weights in the network memory indicates number of vehicles being tracked. 3 Experimental Results The proposed system has been tested with the image of size 64x48 pixels. The camera is installed in the car at 1.2 meter height. The system shows desirable performance in tracking of vehicles. Examples of the tracking are shown in Fig. 11 which also demonstrate the capability of simultaneously tracking multiple vehicles. Fig. 12 displays examples of number of vehicle and error of vehicle tracking in image frames. Examples of network weights which are vehicle feature vectors of vehicles being tracked are depicted in Fig. 13. Table 1 shows overall performance of vehicle tracking system with percent of accuracy over 99%. The percent accuracy is computed from a sequence of 83 images of 1 different environments. Note that the network is capable of tracking both frontal and rear

5 Proceedings of the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES (AIKED '9) ISSN: ISBN: (a) (b) (c) (d) Figure 11: Image samples of vehicle tracking (a) vehicle detection (b) Gabor filtered image (c) saturation image (d) vehicle energy Figure 13: Example of network weights (vehicle feature vectors) of 2 different vehicles being tracked view of the vehicle as can be seen in Fig. 14 and 15. (a) (b) Figure 12: Example of vehicle tracking (a) number of vehicle during the tracking (b) error detection 4 Conclusion The vehicle tracking system has been proposed by using adaptive resonance theory network to recognize the principal component analysis of the Gabor vehicle image. Vehicles within images are initially detected using symmetry energy, Gabor energy, and road energy. Using combination of all energy allows the system to track vehicles more efficiently. Desirable results have been achieved at percent accuracy over 99%. The false detection of vehicles mainly comes from the vehicle detection system. The optimization of energy computation has been developing to overcome this shortcoming. This work, however, has presented the use of unsupervised ART network to track vehicle with the performance that is sufficient to implement in practical system. Acknowledgements: The research was supported by Suranaree University of Technology. References: [1] J. van Leuven, M.B. van Leeuwen and F.C.A. Groen, Real-Time Vehicle Tracking in Image Sequences, Proceedings of IEEE Instru-

6 Proceedings of the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES (AIKED '9) ISSN: ISBN: Table 1: Percent accuracy of vehicle tracking system Data set Total [2] Figure 14: Examples of frontal vehicle tracking [3] [4] [5] [6] [7] Figure 15: Examples of rear vehicle tracking # of frames Missed tracking % Accuracy mentation and Measurement Technology Conference, Budapest, May 21-23, 21, pp K. She, G. Bebis, H. Gu and R. Miller, Vehicle Tracking Using On-Line Fusion of Color and Shape Features, IEEE International Conference on Intelligent Transportation Systems, Washington, DC, October 3-6, 24, pp L. Xie, G. Zhu, Y. Wang H. XU and Z. Zhang, Real-time Vehicles Tracking Based on Kalman Filter in a Video-based ITS, IEEE International Conference on Communications, Circuits and Systems, 25, pp A. Bevilacqua, L. Di Stefano and S. Vaccari, Occlusion Robust Vehicle Tracking based on SOM (Self-Organizing Map), Proceedings of the IEEE Workshop on Motion and Video Computing, Vol.2, 25, pp C. Sotthithaworn, P. Kumsawat, K. Attakitmongcol and A. Srikaew, Vehicle Tracking System Using PCA and Adaptive Resonance Theory, Proceedings of the 7th WSEAS International Conference on Signal, Speech and Image Processing, Beijing, China, September 15-17, 27, pp D. Hoiem, A. Efros and M. Hebert, Putting Objects in Perspective, IEEE Computer Society Conference, 26, pp S. Grossberg, G.A. Carpenter, and D. Rosen, ART 2-A:An Adaptive Resonance Algorithm for Rapid Category Learning and Recognition, IEEE IJCNN-91-Seattle International Joint Conference on Neural Networks, 1991, pp