An approach for monitoring pedestrian by using multi-connect architected MCA associative memory

Transcription

1 International Journal of Digital Content Technology and its Applications Volume 3, Number 2, June 2009 An approach for monitoring pedestrian by using multi-connect architected MCA associative memory Emad Issa Abdul Kareem *1, Aman Jantan *2 *1 Universiti Sains Malaysia (USM), School of Computer science, Penang, Malaysia *2, Corresponding author Aman Jantan, Universiti Sains Malaysia (USM), School of Computer science, Penang, Malaysia doi: /jdcta.vol3.issue2.kareem Abstract The unnecessary long cross road waiting times for pedestrians is due to the inefficient traffic light system configuration and has led to the increasing number of traffic accidents involving pedestrians. This research designs, develops, constructs and tests the pedestrian monitoring approach to elevate the efficiency of traffic lights system configuration. This approach consists of two phases which are the learning and monitoring phase. The monitoring phase involves the use of a single video camera to each street in the intersection. The experiments have shown an increase in the traffic light system configuration as well as the rapid adaptability of the proposed monitoring approach to the environment. Keywords Computer vision, intelligent system, traffic light monitoring system, neural network, associative memory, pedestrians images detection. 1. Introduction Evidently, as stated in reports and statistics, traffic accidents involving pedestrians often occur in the areas which are considered safe. For example, 15% of the total number of people killed on European roads is pedestrians, and another 28% are vulnerable road users. It is stated that most accidents take place in urban areas where serious or fatal injuries can be produced at relatively low speeds, particularly in the case of children [Ángel et. al., 2005]. In order to solve the above mentioned problem, computer vision systems have been applied in traffic systems. Initial experiments developed in 1980 and 1990 put the emphasis in cameras placed near the road or in city streets. However the approach is always performed by human to supervise the scenarios, with operators seeing the images. Gradually, computer vision systems were broadly considered and impelled in two aspects. First, computer vision systems were widely placed in more positions, such as tunnels and other places related to automotive world such as streets, highways, etc. Second, computer vision system were evolved and more complex techniques are used to help human supervision, which includes license plate recognition, counting of cars, detection and tracking of cars, etc. [Jorge et. al.,2004]. The proposed approach has been built based on the human brain principles monitoring technique which uses small associative memory to provide image analysis, and decisions, at real time. Generally, associative memory is a simple singlelayer neural network; which is able to learn a set of pattern pairs (or associations). An efficient associative memory can store a large set of patterns as memories. During recall, the memory is turned on with a key pattern (also called the search argument) containing a portion of information about a particular member of a stored pattern set. This particular stored prototype can be recalled through association of the key pattern and the information memorized [Zurada, 1996], [Fausett, 1994]. The associative memory which is used in this research is multi-connect architecture associative memory (MCA) neural network [Emad, 2004]. MCA works on two principles: first is using the smallest size of net and the second is the learning process which is applicable to parts of the patterns instead of the entire pattern (avoiding learning the similar parts several times). MCA neural network consists of two phases (learning phase and converge phase) [Emad, 2004]. As illustrated in the framework of this research (see figure 1), any intelligent traffic light system consists of two subsystems: monitoring system and control 75

2 An approach for monitoring pedestrian by using multi-connect architected MCA associative memory Emad Issa Abdul Kareem, Aman Jantan system. This research, proposes a novel approach to enhance the monitoring system. This enhancement will increase the efficiency of the traffic light control system because all the intelligent monitor decisions will be provided to it(see figure 1 again). This approach must be able to determine one of two pedestrians cases: 1- NO, no pedestrians waiting to cross the street. 2- Yes, yes there are pedestrians waiting to cross the street. The proposed approach consists of two phases: Learning phase: implementation of the learning process of the two pedestrian cases. Monitoring phase: implementation of the monitoring process of all the streets in the intersection including the process of determining the pedestrian cases with the use of only one video camera on each street. These video cameras will be installed in a suitable place in the street to obtain the best possible view. The experiment presents promising results when the proposed approach applied by using program to monitor one street in an intersection in Penang, Malaysia. The program was able to determine any pedestrians waiting to cross the street depending on the stream of images which are extracted from streets video camera. 2. Background There have been researches presented to propos different approaches to develop an efficient intersection monitoring system. One of these researches is [Ángel et al. 2005] a computer vision systems via a developed intelligent traffic-light system which is capable to capture the presence or absence of vehicles, pedestrians and their particular situations defined by their trajectories. The system consists of two cameras situated at only one signal post placed at an intersection. One of them focuses on the pedestrian crossing while the other one focuses on the vehicles arriving grid. The prototype has been developed to be carried out by a conventional PC. Each camera is connected to a Matrox Meteor II capture card with a resolution of pixels. The images are alternatively taken as both capture cards share the same data bus. The image acquisition and processing is fast enough to make decisions in real time about the traffic light. Pedestrian detection is a fundamental component of image-based traffic monitoring system, and has been implemented through various different approaches. The commonly used approaches for pedestrian detection are gray-level comparison, inter frame subtraction, background subtraction, edge detectionbased method. Figure (1): The framework of this research. A real-time system for pedestrian tracking in sequences of grayscale images acquired by a stationary camera was presented by [Osama et al., 1996]. Integrating this system with a traffic control application such as a pedestrian control scheme at intersections. The system outputs the patio-temporal coordinates of each pedestrian during the period that the pedestrian is in the scene. Processing is done at three levels: raw images, blobs, and pedestrians. Blob tracking is modeled as a graph optimization problem. Pedestrians are modeled as rectangular patches with a certain dynamic behavior. A cooperative approach for detecting and tracking pedestrians in an urban environment was presented by [David et. al., 2004]. Its originality lies in the cooperation of two vision systems. A monocular vision system retrieves feature elements and these elements are visualized. This cooperation supplies a method for detecting pedestrian presence. Then, it allows pedestrian tracking through multiple images. Motion is a strong cue that can be used to classify dynamic scenes and hence detect abnormal movements that can be related to critical situations. 76

3 International Journal of Digital Content Technology and its Applications Volume 3, Number 2, June 2009 [Boghossian et al., 1999] presented research deals with real-time image processing of crowded outdoor scenes with the objective of creating an effective traffic management system that monitors urban settings (urban intersections, streets post-sports events, etc.). The proposed system can detect, track, and monitor both pedestrians (crowds) and vehicles. It describes the characteristics of the tracker that is based on a new detection method. Initially, it produces a motion estimation map. This map is then segmented and analyzed in order to remove inherent noise and focus on particular regions. Tracking of these regions is obtained in two steps: fusion and measurement of the current position and velocity, and then estimation of the next position based on a simple model. The instability of tracking is addressed by a multiple-level approach to the problem. The computed data is then analyzed to produce motion statistics. [Gavrila, 2007] proposed a video-based driver assistance system for the detection of potentially dangerous situations involving pedestrians, in order to either warn the driver, or, if no such time remains, initiate appropriate protective measures (e.g. automatic vehicle braking). The use of video sensors comes quite naturally for this problem; they provide texture information in fine horizontal and vertical resolution, which in turn enables the use of discriminative pattern recognition techniques for distinguishing pedestrians from other static and dynamic objects in the traffic environment. Recently, due to the decreasing cost of infrared devices, the benefits and advantages of using infrared cameras have been actually considered.pedestriandetection systems which have been developed, show that infrared images can facilitate the recognition process. An analysis of color-, infrared-, and multimodalstereo approaches to pedestrian detection was presented by [Stephen and Mohan, 2007]. They were designing a four-camera experimental test bed consisting of two color and two infrared cameras for capturing and analyzing various configuration permutations for pedestrian detection. This four-camera system is incorporated in a test vehicle and comparative experiments of stereo-based approaches to obstacle detection were conducted using unify -modal color and infrared imageries. A detailed analysis of the color and infrared features used to classify detected obstacles into pedestrian regions is used to motivate the development of a multimodal solution of pedestrian detection. In other words, the research proposed a multimodal trifocal framework consisting of a stereo pair of color cameras coupled with an infrared camera. [Massimo et. al., 2004], describes a system for pedestrian detection in infrared images, this system has been implemented on an experimental vehicle equipped with an infrared camera. It is based on a multi-resolution localization of warm symmetrical objects with specific size and aspect ratio. Because road infrastructures and other road participants may also have such characteristics, a set of matched filters is included in order to reduce false detections. A final validation process, based on human shape s morphological characteristics, is used to build the list of pedestrian appearing in the scene. Method for detecting pedestrian was recently implemented on the ARGO vehicle ARGO is an experimental autonomous vehicle equipped with vision systems and featuring automatic steering capability. An approach is described by [Broggi et al, 2000], which exploits the morphological characteristics and the strong vertical symmetry of the human shape for the detection and recognition of pedestrians. The perception of the environment is performed through the processing of images acquired from a vision system installed on board of the vehicle. The analysis of a monocular image delivers a first course detection. Finally, neural networks also have been successfully applied to many real-time intelligent vehicle systems. [Liang et. al., 2000] used a neural network and trained it with the backpropagation algorithm. He presented an algorithm for detecting pedestrians in a cluttered scene from a pair of moving cameras. This is achieved through stereo-based segmentation and neural network-based recognition. Next section 2.1 will be illustrate multi-connect architecture associative memory (MCA). 2.1 Multi-connect architecture MCA associative memory Inspired by the structure of the human brain, artificial neural networks have been widely applied to different fields, because of their ability to solve cumbersome or intractable problems by learning directly from data [Emad et. al., 2006]. MCA is iterative auto-associative networks consisting of a single layer of fully connected processing elements, thus it is categorize as an associative memory. MCA has three processing elements which include all the processing elements (y1, y2, y3: neurons or nodes) are connected in feedback architecture with the multi-layer connection weights specified in a certain way, see figure (2) [Emad, 2001]. 77

4 An approach for monitoring pedestrian by using multi-connect architected MCA associative memory Emad Issa Abdul Kareem, Aman Jantan t: limiting (threshold) value, which equal to zero in MCA. In the vector form the energy function is written as: E = 1 v. w. v + t. v 3 2 or with t=0 we have: ( ) ( ) Figure (2): MCA architecture. y1, y2 and y3 are elements process. M: number of connections (paths) between two elements process Where m<=8 This net dealing with unlimited Pattern sizes (pattern: means sequence of 1's and 0's). The net for bit plane k consists of 3 neurons. The feedback input to the i th neuron is equal to the weighted sum of neuron outputs vj. wij is the weight value connecting the output of the j th neuron with the input of the i th neuron. The total input net i of the i th is as follows [Emad, 2004]: n k net i = Wi V j + ii ( 1) j i= 1 Note this net uses the number of neurons here n=3, which is the simplest and most efficient form of associative memory as used by [Emad, 2004]. Additionally, the number of neuron connections (paths) in this case will be 2 3 = 8, (see figure 2). 2.2 Energy function MCA is given by weights and limiting values, then the network will be in dynamic equilibrium when one creates a pattern. A network can be defined by various patterns; one can find them by the different start vectors in the iteration. Corresponding to the spin glass theory of solid state physics, such equilibrium functions in Hopfield networks are characterized by the fact, that the total energy (Hamilton function) becomes minimum. This leads here to a Lyapunov function or energy function, which becomes exactly minimum value, when one creates a pattern. This energy function can be defined as follows [Werner, 1995]: 1 n n n E = 2 wijviv j + tkv i= 1 j= 1 k = 1 k ( 2) Where n: the number of elements in the vector v. wij: the weight from the output of neuron i to the input of neuron j. E = 1. v.( w. v) (4) 2 One can calculate the energy function E for every input vector, which can be created in the network. If one calculates the function E for all the possible input vectors, an energy landscape with maximums and minimums can be obtained. The point is that the minimum is taken when the input is a pattern. 3. Intelligent Monitoring approach for Pedestrians Using (MCA) This research will be adapting MCA for intersection monitoring system for pedestrians. There are some concepts about image representation needs to show before going on further into details.. Image representation is concerned with the representation of input data, which can be measured from the objects that can be recognized. The input data is acquired by digitizing images, which are obtained through scanner devices [Julius and Rafael, 1974], [David, 1999]. The digital image is represented as two dimensional array of data I(x, y), where each pixel value corresponds to the brightness of the image at the point (x, y). In linear algebra terms, a two-dimensional array is referred to as a matrix, and one row (or column) is called a vector [Emad et. al., 2006]. Normally there are many types of image data which include color, gray-level and Binary images. For color images each pixel value corresponds to the brightness and color information. But gray-level images are referred to as monochrome images, and they only contain brightness information only, i.e. no color information [Emad et. al., 2006]. Finally binary images are the simplest type of images and are able to take one of two values, typically black and white, or 0 and 1. A binary image is referred to as 1 bit/pixel image because it takes only 1 binary digit to represent each pixel [Emad et. al., 2006]. These types of images (binary images) are most frequently used in computer vision applications where 78

5 International Journal of Digital Content Technology and its Applications Volume 3, Number 2, June 2009 the only information required for the task is general shape, or outline information[emad et. al., 2006]. The proposed monitor approach will use one video camera in every street of the intersection. These video cameras will be installed in a suitable place in the street, to obtain the best possible view. The camera monitors the expected waiting area of pedestrians in the street (see figure (4)). Each street camera in the intersection sends its own video stream. This video will be converted to stream of JPG images Evidently, MCA deals with the bipolar system (i. e. -1 and +1) for direct input data, so it is useful for binary images, but not useful for gray-level or color images unless there is another way for input data of such images. To overcome this obstacle, this research used one of edge detection filters (the most important tasks of computer vision). An edge marks the border of an object that is characterized by a different feature (gray value, color, or any other property) than the background. In the context of simple neighborhoods, an edge is a special type of simple neighborhood with a sharp transition. Low-level edge detection thus means to detect the strength of such a transition and the direction of the edge [Bernd and Horst, 2000]. 1 x > 0 f ( x) = ( 4) 1 x 0 The action of the proposed monitor approach divided by two phases: learning phase and monitoring phase. Learning Phase: there are two cases to be considered: 1- NO, no pedestrians waiting in to cross the street. 2- Yes, yes there are pedestrians waiting to cross the street. In this phase, learning process will be implemented to learn these two street cases to acquire the ability to recognize these cases. Learning phase will be used training images to learn street cases. For each case, learning process will use at least one training image. As mentioned before, the proposed approach will re-present the training images as black and white (monochrome) images using edge detection filter. These images will provide visual information which will be processed to extract data of the possible cases of pedestrians in each street in the intersection. This phase will end through the implementation of learning process to learn the pedestrians cases using multi-connect architecture associative memory and save all the visual information(see figure(5)). Monitoring Phase: In this phase, monitoring process will be implemented to monitor an intersection depending on the visual information which is already saved in the associative memory during learning phase. Each street camera in the intersection sends its own video stream. This video will be converted to stream of JPG images. Just like previous phase, the proposed approach will presents the images as black and white (monochrome) images using edge detection filter. These images provide visual information to determine the case of pedestrians in each street in the intersection. These determinations are completed after the convergence of these images inclusive of the training images saved in the associative memory (see figure (6)). Figure 4: View from the top and the side of the street. Then, The age detection filter processing was implemented and by using hard limiter function (see equation 4), the output binary image is converted to bipolar data[fausett, 1994]. 79

6 An approach for monitoring pedestrian by using multi-connect architected MCA associative memory Emad Issa Abdul Kareem, Aman Jantan Training Images Video Cameras Capture Learning Phase Monitoring Phase Video from Cameras Edge Detection filter Stream of Images with Black and white Convert Video to Images Multi-Connect Architecture Neural Network Learning process Associative memory Edge Detection filter Stream of Images Figure (5): learning phase of the proposed. 4. Experiment and result analysis This experiment intends to illustrate the proposed approach and proving it by writing the program. This program will be written in Delphi language. Empirically, in this experiment, we have 43 JPG images which are extracted from video sample. This video sample taken from an intersection in Penang, Malaysia (see figure (9)). At first we started with the learning phase, the program learned one training image for the background of the expected waiting area of the pedestrians as NO case (no pedestrian waiting in this street). As mentioned before, the program presents the training image as black and white image using edge detection filter (see figure 6). To finish learning phase, the program starts to learn this black and white training image (see figure 7). Then, the second phase (monitoring phase) will be implemented to test the ability of the program to recognize the entire 43 JPG images 300*400 pixels (see Figure 9). The program shows its ability to recognize all the 43 images using just one training image (see Figure 8). Based on the fact that the pedestrian case in any street will be one of two cases, first case is: no pedestrians waiting in the street and second case is: yes there are pedestrians waiting in the street, thus the experimental program just need to learn one of this two cases which it is the first case. When this program cannot determine the first case this means the pedestrians case is the second case. Multi-Connect Architecture Neural network converge process Stream of Images with Black and white Associative memory Figure (6): Monitoring phase of proposed approach. Figure (6): Convert jpg image to black and white image using edge detection filter. 80

7 International Journal of Digital Content Technology and its Applications Volume 3, Number 2, June 2009 In addition, depending on the limited pedestrians cases, the proposed approach acquired the adaption facility to its environment rapidly. Logically, the learning process will be implemented just one time using small associative memory to gain the required experience. Figure (7): learn black and white image. Figure (8): The program determines the waiting pedestrians. 5. Conclusions Comparing with the previous computer vision monitoring approaches this research has similar principles except that it uses MCA with the proposed approach. Thus, unlike other approaches, learning process will be learn each training image as one segment, instead of using any pedestrians detection, object tracking or even image segmentation techniques. The associative memory gives the proposed approach the ability of adaptation to any street. The observation points out to the importance of the accurate choosing of the training images and its number to exploit the adaptation feature. Figure (9): 12 JPG selected sample from 43 tested JPG images 300*400 pixels. 6. References [1] M. B. A. Broggi, A. Fascioli and M. Sechi, Shapebased Pedestrian Detection, Proceedings of the IEEE Intelligent Vehicles Symposium, [2] C. C. Ángel Serrano, Licesio J. Rodríguez, Aragón, Raquel Montes and Enrique Cabello, Computer Vision Application: Real Time Smart Traffic Light, SpringerVerlag Berlin Heidelberg,

8 An approach for monitoring pedestrian by using multi-connect architected MCA associative memory Emad Issa Abdul Kareem, Aman Jantan [3] D. H. Chang, Fingerprint Recognition Through Circular Sampling, Center for Imaging Science, Rochester Institute of Technology, [4] S. e. M. David Lef ee, Abdelaziz Bensrhair and Massimo Bertozzi, Cooperation of Passive Vision Systems in Detection and Tracking of Pedestrians, IEEE Intelligent Vehicles Symposium, [5] Emad Issa Abdul Kaream, Alternative Hopfiled Neural Network With Multi-Connect Architecture, journal of College of Education, Computer Department, Almustansiryah University, Baghdad, Iraq, [6] Emad Issa Abdul Kareem, Hopfield Neural Network Using Genetic Algorithm, M.Sc. thesis, High studies institute for computer and information, Baghdad, Iraq, [7] k. N. M. Emad Issa Abdul Kareem, Hussein A. Moussa, Gray Image Recognition Using Hopfield Neural Network with Multi- Bit plane and Multi-Connect Architecture, Proceedings of the international Conference on Computer Graphics, Imaging and Visualization (CGIV'06) IEEE, [8] L. Fausett, Fundamental of Neural Networks, Architectures, Algorithms and Applications, Prentice- Hall [9] D. M. Gavrila, Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle, International Journal of Computer Vision, Springer Science + Business Media, LLC. Manufactured in the United States [10]J. T. T. a. R. C. Gonzalez, Pattern Recognition Principles, Addison-Wesley publishing company, [11] B. J. a. H. Haußecker, Computer Vision and Applications: printed by Academic Press, [12]P. G. Jorge Pérez, Jesús Silva, Enrique Cabello, Jesús Monclús and Tomás Santa Cecilia Cristina A Conflict- Avoiding, Artificial Vision Based, Intelligent Traffic Light Controller, Universidad Rey Juan Carlos, ESCET, C/ Tulipán S/N,28933 Móstoles (Spain), (2) RACE,C/ Isaac Newton 4,28760 Tres Cantos (Madrid), [13]W. Kinnebrock, Neural Network, Fundamentals, Applications, Examples, Galotia Publications, [14] a. C. E. T. Liang Zhao, Stereo and Neural Network-Based Pedestrian Detection, IEEE Transactions on Intelligent transportion Systems, vol. 1, [15]Massimo Bertozzi, Alberto Broggi, Alessandra Fascioli, Thorsten Graf and Marc-Michael Meinecke, Pedestrian Detection for Driver Assistance Using Multiresolution Infrared Vision, IEEE Transactions on Vehicular Technology vol. 53, [16]S. J. K. a. M. M. Trivedi, On Color-, Infrared-, and Multimodal-Stereo Approaches to Pedestrian Detection, IEEE Transsctions on Intelligent Transportation Systems, vol. 8, [17]B. A. B. a. S. A. Velastin, Image processing System for Pedestrian Monitoring using neural classification of normal motion patterns, Measurement and Control, vol. Volume 32, no. Issue 9 (Special Issue on Intelligent Vision Systems ) [18] J. M. Zurada, Introduction to Artificial Neural Systems, Jaico Publishing House,