Real-time Traffic Congestion Detection Based on Video Analysis

Journal of Information & Computational Science 9: 10 (2012) 2907 2914 Available at http://www.joics.com Real-time Traffic Congestion Detection Based on Video Analysis Shan Hu a,, Jiansheng Wu a, Ling Xu b a School of Software, University of Science and Technology Liaoning, Anshan 114051, China b School of Resources and Civil Engineering, University of Science and Technology Liaoning Anshan 114051, China Abstract To detect the real-time information of the traffic congestion on the road, a method based on real-time video analysis was presented. First of all, the density of the vehicles on the lane as an important factor in traffic congestion was obtained by dynamic information of the moving vehicles on lane. Secondly, the speed of vehicles is calculated by the optical flow field on the corner points of vehicles. Finally, judges the current condition of the traffic flow by fuzzy logic based on the conditions of density and velocity of the vehicles. The proposed method is able to accurately and timely detect the status of traffic congestion. Keywords: Congestion Detection; Frame Difference; Moving Locus; Lane Detection; Corner Point; Optical Flow; Fuzzy Logic Decision 1 Introduction With the fast development of automobile industry, traffic congestion becomes a troublesome problem that perplexes many modern cities around the world. To reduce negative effect of traffic congestion, Intelligent Transportation Systems (ITS), as an efficient way to solve traffic congestion problem, are currently being paid more attention to by more and more researchers. In order to design an efficient traffic management system, ITS should obtain one critical capacity to effectively monitor the traffic conditions and rapidly detect traffic congestion. Maurin [1] designed a system which addresses a multi-level approach to monitoring traffic scenes using the technologies of optical flow, Kalman filtering, and blob merging. Yu [2] proposed a tracking based algorithm that extracts traffic information from compressed video and uses the ratio between the moving blocks and all blocks to estimate traffic situation. Chan [3] used dynamical texture model to identify the traffic scene. Porikli [4] presented a traffic congestion estimation method that directly extracts features in the MPEG compressed domain with HMM. However most of these systems are either designed for specific camera setups or computationally Project supported by Department of Education of Liaoning Province (No. L2010202) Corresponding author. Email address: samhoo74@126.com (Shan Hu). 1548 7741 / Copyright 2012 Binary Information Press September 15, 2012

2908 S. Hu et al. / Journal of Information & Computational Science 9: 10 (2012) 2907 2914 expensive. In this paper, we present a novel approach for detecting condition of traffic congestion based on real-time video. The paper is organized as follows: in Section 2, we discuss the method that could detect the density of vehicles on lane, followed by the proposed algorithm that could calculate vehicle velocity based on the optical flow vector of the corner points on vehicles in Section 3. In Section 4, we present fuzzy logic judgment system based on density and velocity of vehicles. 2 Density Detection Based on Information of the Moving Vehicles The density of vehicles is an important factor in the detection of traffic congestion. Sometime, we can use the density of vehicles on lane to judge the degree of congestion. In this paper, the density of vehicles is detection by following procedure: 1) Moving vehicles detection based on frame difference 2) Lane detection based on accumulating vehicles moving locus. 2.1 Moving Vehicles Detection Based on Difference Between Frames Frame-difference methods [5, 6] have been used widely for moving target detection. In this work, frame-difference methods were employed to detect the moving region of the moving vehicles. The steps of the algorithm that detect the moving vehicles as follow: 1) Change the RGB image to gray image 2) Subtract the two adjacent frames; 3) Employ the median filter to get rid of noise points; 4) Check the difference result by Eq. (1) to get an binary image; 5) Apply the morphology operation of close to process the binary image And fill region of vehicles; Diff n (x, y) = { 1 if It (x, y) I t+1 (x, y) > T 0 others (1) where I t (x, y) and I t+1 (x, y) are the gray value of spatial location (x, y) in frame t and frame t + 1. T is a experimental threshold, here we choose 30 as the threshold. The result of the above algorithm is showed in Fig. 1. 2.2 Lane Detection Based on Moving Locus of Vehicles With the development of ITS based on vision, many methods to detect the lane region spring up. Currently, the methods to detect lane region mainly focus on structured lanes and unstructured lanes. For structured lanes, the localization of lane borders or lane markings is one of the most commonly used approaches. Color cue [8], Hough transform [7, 10], Spline model [9] etc. have been

S. Hu et al. / Journal of Information & Computational Science 9: 10 (2012) 2907 2914 2909 Fig. 1: Detected moving vehicles. Left image is original image. Right image shows the result of detected vehicles on the lane. utilized to find the road boundaries or markings. The drawbacks of these methods are that they only consistently work for structured roads with noticeable markings or borders. For unstructured lane, lane detection methods primarily take lane features into account. These methods catch key point on color, intensity or texture information between lane region and the non-lane region [11], are usually insensitive to road shapes, but sensitive to illumination, shadows and water marks. In this paper, we take a new and easy way to detect lane by tracking moving locus of vehicles. In order to detect lane, the region of the lane will automatically been created by the way of accumulating the locus of moving vehicles. The lane-detection algorithm based on accumulation of the moving vehicles locus mainly involves two procedures: 1) Accumulating the region of moving vehicles. Due to the locus of moving vehicles is part of lane, the region of the whole lane can be got by continuously accumulating the locus of moving vehicles. The process of the accumulating moving vehicles locus is shown by Fig. 2. 2) Using the morphology operation of close to concatenate the area as an entirety region. Sometimes, there are some small holes in the region of lane that is accumulated by moving vehicles locus. However, the morphology operation close can be used to fill in particular regions of an image. The result of close operation on region of accumulated moving vehicles locus is shown in Fig. 3. Accumulating 3 frames Accumulating 7 frames Accumulating 15 frames Accumulating 25 frames Fig. 2: Procedure of accumulating moving vehicles locus 2.3 Vehicle Density Detection After acquiring the moving vehicles binary image, it is easy to compute out the area of moving vehicles A vehicles on the lane through counting the number of pixels that represents the vehicles. In the same way, the area of lane A lane can be represented by lane binary image. The density of

2910 S. Hu et al. / Journal of Information & Computational Science 9: 10 (2012) 2907 2914 Fig. 3: Result of lane detected by accumulating locus vehicle on the lane can been figured out by formula as following Eq. (2): DEN = A vehicles /A lane (2) 3 Speed Estimation Based on Optical Flow Field In order to estimate the speed of moving vehicles, velocity vectors of a sufficient number of reference points identified on the vehicle must be figured out. This procedure involves three main steps. Firstly, a sufficient number of feature points from the vehicle are selected, and these points must be accurately tracked on at least two successive video frames. Secondly, the velocity vectors of feature points are computed based on optical flow estimation. 3.1 Feature Points Detection In order to track moving vehicles within video frames, points to be tracked which belong to the vehicles on the successive video frames, should be selected automatically. It is well known that good features to be tracked are corner points which have large spatial gradients in two orthogonal directions. One of the most frequently used definitions of a corner point is given in [12]. This definition defines a corner point by a matrix which is expressed by second order derivatives. The Hessian matrix in 2D space is defined by Eq. (3): 2 I x 2 2 I y x 2 I x y 2 I y 2 (3) In our system, as soon as the camera begins to acquire image, corner points are selected continuously in real time from the frame images. On the first frame, points are selected and on the next frames those points are tracked and instantaneous velocity vectors of those points are computed. In our system, we restrict the number of points between 20 and 100 and at least 20 points are selected in the worst case.

S. Hu et al. / Journal of Information & Computational Science 9: 10 (2012) 2907 2914 2911 3.2 Optical Flow Vector Calculation For speed estimation, correspondence of each selected point on the first frame on which the vehicle appears for the first time, must be found on the next frame. In order to find the corresponding point, there is no prior information other than the point itself, and it seems that it is not also possible to find exactly where the match point is. However, if we assume that each image in the each frame is flowing by the very short time period and thus changing the position during the flow, then a modeling approach which models this flow event can be used. These kinds of flow model are called optical flow [13, 14]. The optical flow can been calculated based on the assumption that the intensity values of a selected point p(t) and its neighbors on the frame image I(t) do not change on the next frame I(t + t), where t is a very short time period. So the assumption can be written in mathematical form as follows Eq. (4): I (p (x, y, t), t) = I (p (x + x, y + y, t + t), t + t) (4) where I(p, t) is the intensity value of a point p on the image I(t). Eq. (4) says that the intensity of the point p on the current image frame does not change during the time period t that passed. In the more mathematical sense, change rate of I(p, t) is zero over time period t, this last situation is formally written as follows: I (p (x, y, t), t) t = I (p, t) p (t) + I p t t = 0 (5) In Eq. (5), the derivative di/dp is spatial derivative at point p on the image frame I(t). Thus it can be expressed by di/dp = di. We can write this expression in explicit form as follows: p (t) t = t (xi + yj) = x t i + y t j (6) It can easily be seen that the vector ( x/ t)i is equal to the velocity of the point p in the x-axis direction. In other words, it is the x component namely v x component of the velocity vector v. In similar way, the vector ( y/ t)j is the y component namely v y component of the velocity vector v. Now we can rewrite the Eq. (6) as follows: p (t) t = v = v x i + v y j (7) It is seen that the derivative I/ t = I t expresses the change rate of the intensity values at point p, between the frame images I(t) and I(t + t) Eq. (5) can be rewritten as follows: Iv + I t = 0 (8) Then Eq. (8) can be written as: [ I x I y ] [ v x v y ] = I t (9)

2912 S. Hu et al. / Journal of Information & Computational Science 9: 10 (2012) 2907 2914 The values of I x, I y and I t in Eq. (9) can easily be computed from the frame images. The variables v x and v y are two unknown components of the velocity vector v and these are respectively the components in the directions x and y axes of the image coordinate system. In Eq. (9), we have two unknowns to be solved, but we only have one equation. Since only one equation is not enough for unique solution of the unknowns, at the moment it seems not possible to solve these unknowns. In order to solve these two unknowns, we need more independent equations. For this purpose, the third assumption of the Lukas-Kanade algorithm is used. That is, point p behaves together with its neighbors. So its neighbors must also satisfy the Eq. (9). In other words, neighbor points (or pixels) of point p must move with the same velocity v(v x, v y ). According to these explanations, the same equations as (9) are written for 3 3 or 5 5 neighborhood of the point p. In this case, we totally have 9 or 25 equations with the same unknown v x and v y. Now the unknowns can be solved with over determined set of Eq. (9) by using least squares or total least squares estimation method. The result of the optical flow velocity on the corner point that is estimated by Lukas-Kanade Optical Flow Method is show as Fig. 4. Fig. 4: The optical flow velocity on the corner point of vehicles 3.3 Velocity of Vehicles Calculation In general, there are many vehicles on lane at the same time. There is no need to take into account all velocity vectors on vehicles. In this system, the greatest 20 velocity vectors of corner points on vehicles were selected as the sample velocity vectors to express the current status of vehicle velocity on lane. The average of these 20 velocity vector can be written as Eq. (10) V mean = v /20 (10) In order to normalizing the velocity, the current velocity status can be written as Eq. (11) V = V mean /V max (11) The V max in Eq. (11) is the maximum velocity vector when vehicle run at maximum velocity on lane. 4 Fuzzy Logic Decision As pointed out in [15], the use of definite boundaries between traffic categories may lead to high error rates when operating near the categories boundaries. This is due to the intrinsic nature of

S. Hu et al. / Journal of Information & Computational Science 9: 10 (2012) 2907 2914 2913 the traffic flows whose variations (e.g. from free flow to traffic jam) follow a continuous process [16], and consequently no clear boundaries exist between the distinct traffic flow statuses. This uncertainty in the definition of the boundaries between different categories makes fuzzy-logicbased systems especially suitable for addressing complex non-deterministic decision problems such as the identification of traffic congestion. To identify the real-time condition of the traffic flow, we use the fuzzy logic system to detect the state of the current traffic flow by the input variable: density of the vehicles on the lane and velocity vector estimated through optical flow equation. As in any fuzzy-based decision system, the input variables are first classified into different fuzzy sets. The possible fuzzy sets for the speed are very slow, slow, medium and fast. For traffic density, the defined fuzzy sets are low, medium, high and very high. One of the main particularities of fuzzy logic is that a fuzzy set can contain elements with partial degree of membership, and consequently, an input value can belong to several fuzzy sets at the same time. In order to determine the degree of membership of the input values to each of the fuzzy sets, membership functions are employed. The membership functions used in this system are illustrated in Fig. 5 (a) and Fig. 1 (b). In addition, output fuzzy sets corresponding to the different traffic categories have also been defined with free-flow=0, slight congestion=1/3, moderate congestion=2/3 and severe congestion=1. To finalize the definition of the fuzzy-based congestion detection system, fuzzy rules that relate the input (speed and density) and the output fuzzy sets (congestion levels) have been established and are displayed in Table 1. The output of the traffic jam quantification system is a continuous value within the interval [0, 1] indicating the level of congestion (traffic jam intensity), with 0 representing free flow and 1 severe traffic congestion. 1.0 low medium high veryhigh 1.0 veryslow slow medium fast 0.5 0.5 0 0 0 0.2 0.4 0.6 0.8 1.0 0 (a) Traffic density membership function 0.2 0.4 0.6 0.8 1.0 (b) Speed membership functions Fig. 5: Input variable membership functions Table 1: Fuzzy rules relating input speed and density sets with output congestion condition Congestion level Traffic density low medium high very high Speed Very slow slight moderate serious serious slow slight slight moderate serious medium free slight moderate moderate fast free free slight moderate

2914 S. Hu et al. / Journal of Information & Computational Science 9: 10 (2012) 2907 2914 5 Conclusion Real-time and accurate traffic condition on the road is important information to intelligent transport system. To realize the real-time and reliable traffic congestion information acquisition system, the computational complexity is an important consideration. The presented method in this paper provides a easy way to obtain the density of lane and velocity vectors of vehicles. At the same time, this method that uses the density of lane and velocity vector of vehicle as input variable for fuzzy logic decision system can both enhance accuracy of system and reduce verdict time. The accurate and timely traffic congestion information can been used to control the signal light at road junctions and also can been sent to driver to avoid recurring congestion. References [1] N. Vasconcelos, Classification and retrieval of traffic video using auto regressive stochastic processes. Proceedings of 2005 IEEE Intelligent Vehicles Symposium, 771-776 [2] B. Maurin, O. Masoud, N. Papanikolopoulos, Monitoring crowded traffic scenes. IEEE Int l Conference on Intelligent Transportation Systems, 2002, 19-24 [3] X. D. Yu, L. Y. Duan, Q. Tian, Highway traffic information extraction from Skycam MPEG video. IEEE Int l Conference on Intelligent Transportation Systems, 2002, 37-42 [4] Fatih Porikli, Xiaokun Li, Traffic congestion estimation using HMM models without vehicle tracking, 2004 IEEE Intelligent Vehicles Symposium, 188-193 [5] R. T. Collins, A. J. Lipton, T. Kanade et al., A System for Video Surveillance and Monitoring, VSAM Final Report. Pittsburgh Carnegie Mellon University. CMU-RI-TR-00-12, 2000 [6] C. Zhan, X. Duan, S. Xu et al., An improved moving object detection algorithm based on frame difference and edge detection. Proc. of the 4th International Conference on Image and Graphics, 2007, 519-523 [7] J. B. Southhall, C. Taylor. Stochastic road shape estimation. ICCV, 2001, 205-212 [8] Tsung-Ying Sun, Shang-Jeng Tsai, V. Chan. HSI color model based lane-marking detection. IEEE Intelligent Transportation Systems Conference, 2006, 1168-1172 [9] Y. Wang, E. K. Teoh, D. Shen. Lane detection and tracking using b-snake. Image and Vision Computing, 22(4), 2004, 269-280 [10] B. Yu, A. K. Jain. Lane boundary detection using a multiresolution hough transform. ICIP, 1997, 748-751 [11] Morten Rufus Blas, Motilal Agrawal, Aravind Sundaresan, Kurt Konolige, Fast color/texture segmentation for outdoor robots. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems Acropolis Convention Center Nice, France, Sept. 2008, 22-26 [12] C. Harris, M. Stephens, A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, Manchester, England, 31 Aug.-2 Sept., 1988, 147-151 [13] B. K. P. Horn, B. G. Schunck, Determining optical flow. Artif. Intell. 17, 1981, 185-203 [14] B. D. Lucas, T. Kanade, An iterative image registration technique with an application to stereo vision. In Proceedings of the DARPA Image Understanding Workshop, Washington, USA, Jun. 1981, 121-130 [15] X. Binglei et al., Fuzzy-logic-based traffic incident detection algorithm for freeway. Proc. of the 7th ICMLC, Kunming, Jul. 2008, 1254-1259 [16] J. Lu, L. Cao, Congestion evaluation from traffic flow information based on fuzzy logic, Proc. of the IEEE International Conference on ITS, Shanghai, Oct. 2003, 50-53