Vehicle Detection and Tracking in Video from Moving Airborne Platform

Transcription

1 Journal of Computatonal Informaton Systems 10: 12 (2014) Avalable at Vehcle Detecton and Trackng n Vdeo from Movng Arborne Platform Lye ZHANG 1,2,, Hua WANG 3, L LI 2 1 School of Traffc and Transportaton Engneerng, Changsha Unversty of Scence and Technology, Changsha , Chna 2 School of Economcs and Management, Tongj Unversty, Shangha , Chna 3 School of Transportaton Engneerng, Tongj Unversty, Shangha , Chna Abstract Vehcle detecton and trackng s one challenge n collectng vehcle trajectores from Unmanned Aeral Vehcle (UAV). When the feature pont cannot be well tracked because of the low qualty of the vdeo, many current algorthms do not work well. To solve ths problem, a vehcle detecton and trackng method was proposed wth enhancements based on prevous studes n several aspects: (1) an accurate vehcle boundary detecton method usng random walker segmentaton to mprove vehcle poston accuracy, (2) a quck partcle flter vehcle trackng method to handle pont feature trackng low qualty problem. To test the performance of the proposed method, two feld experments have been conducted. The detecton and trackng rates of these experments are 88% and 90% respectvely, whch s hgher than the prevous algorthm. Keywords: Vehcle Trajectory; Vehcle Detecton; Partcle Flter Trackng; Unmanned Aeral Vehcle; Random Walker 1 Background Vehcle trajectory data s the ultmate emprcal data to nvestgate drvng behavor. A wde range of vehcle trajectory data collecton methods have been developed usng sensng technologes ncludng aeral photography, vdeo, mcrowaves and the Global Postonng System [1]. Amount those method, traffc cameras mounted on arborne platform (e.g., helcopters) has been consdered to be a promsng method [2]. Compared wth vdeo cameras mounted on traffc lghts or poles, arborne camera could cover large road extent and flexble to employ, whch could capture the hgh-resoluton spatal-temporal traffc nformaton. Project supported by the Natonal Hgh Technology Research and Development Program of Chna (No. 2009AA11Z220). Correspondng author. Emal address: chnazhangly@126.com (Lye ZHANG) / Copyrght 2014 Bnary Informaton Press DOI: /jcs9729 June 15, 2014

2 4966 L. Zhang et al. /Journal of Computatonal Informaton Systems 10: 12 (2014) Vehcle detecton and trackng s one challenge n usng arborne platform for traffc nformaton collecton, because of the moton of both camera and the platform. Wth the advances of computer vson technology, several vehcle detecton and trackng methods are developed to extract vehcle trajectory for traffc flow study [2, 3], traffc flow montorng [4, 5] or traffc ncdent detecton [6]. In the feld of computer vson, several methods have been proposed to enhance vehcle detecton and trackng, whch are sutable for specfc scenaro. When the feature pont cannot be well tracked (e.g., very few feature ponts on a vehcle could be tracked correctly), because of the low qualty of the vdeo, many current algorthms do not work well. Furthermore, n order to get the accurate traffc nformaton vehcle boundary should be detected accurately durng vehcle detecton, whch s neglected n most of the current studes. So, the research ams of the paper s to develop a vehcle detecton and trackng method for arborne vdeo n whch feature pont trackng qualty s low, whch can detect the vehcle boundary precsely. The remander of the paper s organzed as follows. Secton 2 provdes a bref summary of the relevant works. Followng ths, the detals of the vehcle detecton and trackng algorthm s fully descrbed. In order to evaluate the performance of the proposed algorthm, feld experments were conducted, and detals of the results are descrbed n Secton 4. Fnally, conclusons and recommendatons for future work are presented. 2 Lterature Revew In the past decades, many vehcle detecton and trackng algorthms have been studed for arborne vdeo processng. Although some of the challenges stll exst, the technologes avalable today allow researchers to overcome many of them. Hoogendoorn et al. [3] proposed a method of detectng vehcles by comparng pxel values n a pcture wth the medan value of that pxel over tme. A detecton rate of 94% s clamed for a feld experment under normal weather condton. Ths method works well for geo-statc helcopter, but not sutable for crusng one. A smlar methodology was stated by Angel and Hckman [5]. Intensty was compared and a concentraton of devatng pxels s marked as a vehcle. The man problem ndcated here was n matchng vehcles, wth a matchng percentage of about 90% reported [5]. Ths method would not be vald for statc vehcles detecton. Shastry and Schowengerdt proposed a smlar computer vson system, whch uses a robust vdeo-regstraton method to correctng for arborne platform moton [7]. The estmated parameters are wthn 10% of manual measurements. Dfferent from [3, 5], Knoop proposed a new method to process arborne vdeo data [2]. The vdeo data are transformed so that the trajectores of the vehcles become vsble n a sngle mage, n whch trajectores can be found by detectng lnes. Wth ths method, about 95% of the trajectores are detected correctly. One constran of ths method s that only one lane could be processed n a longtudnal cross-secton mage [2]. Many studes on UAV vdeo processng have been reported. Renartz and Lachase proposed an UAV vdeo processng algorthm usng dgtal road database to mprove the performance [8]. Cao [9] proposed a vehcle detecton and trackng method based on mult-moton layer analyss, whch works well when pont features could be well-tracked. In [10], a vehcle detecton method usng dynamc Bayesan networks was proposed. Ths method should be properly traned before deployment. Most of these methods just detect vehcles boundary roughly whch s not meet the need for precse vehcle poston and vehcle sze detecton.

3 L. Zhang et al. /Journal of Computatonal Informaton Systems 10: 12 (2014) In a word, there s no vehcle detecton and trackng method whch works well n most of the scenaros and vehcle detecton and trackng s stll a challenge n vdeo processng. It s meanngful to study the robust vehcle detecton and tracng method for low qualty vdeo from crusng arborne platform, whch can detect the precse vehcle boundary. 3 Vehcle Detecton and Trackng Method Vehcle detecton and trackng n vdeo from movng UAV s a challengng. Some methods are only sutable for geo-statc UAV [3], and some s dffcult to process multple lanes [2]. In our prevous study [6], we proposed a vdeo processng method. But when the vbraton of the UAV s serous and the vdeo qualty s low, the vehcle trackng method does not work well. Furthermore, n order to get accurate vehcle trajectory and vehcle sze, vehcle boundary should be detected precsely. To resolve these problems, a new method s proposed, whch s mproved n the follow aspects. Frst, a new vehcle boundary detecton method usng random walker segmentaton was adopted to mprove the accuracy of vehcle poston and sze. Second, a quck partcle flter method was proposed for vehcle trackng, whch can track vehcles even n low qualty vdeo. The process s llustrated n Fg. 1 and the mentoned mprovements are descrbed n follow sectons. Fg. 1: Flow chart of vdeo processng method 3.1 Robust mage regstraton Image regstraton s crucal for arborne vdeo processng, whch could elmnate the moton of the camera. Many algorthms have been proposed, such as the block based algorthms [11] and feature pont based algorthms [7]. Snce only the feature ponts not located on movng vehcles

4 4968 L. Zhang et al. /Journal of Computatonal Informaton Systems 10: 12 (2014) could be used for mage regstraton, a wndow-szng method s adopted n [7]. One constran of ths method s that the wndows sze should be set properly by users. In ths paper, we use an mage regstraton algorthm smlar to the one proposed n [7]. We use SURF (Speeded-Up Robust Features) algorthm for feature pont trackng [12] and adopted a cluster method to select feature ponts not on movng vehcles. The algorthm s as followng: Input Two consequent gray mage I and I +1. Output The wrapped mage I T. Step 1 Fetch two consequent mage I and I +1 from vdeo. Step 2 Get tracked feature ponts between I and I +1, noted as P and P +1 respectvely. Where N s the number of feature ponts tracked. P = {p 1, p 2,...p N } (1) P +1 = {p +1 1, p 2 +1,...p N +1} (2) Step 3 Calculate movement of all trackng feature ponts V ; Select feature ponts on background of I and I +1 accordng to V, noted as P B and P+1. B Note feature ponts on movng objects n I and I +1 as P F and P+1 F respectvely. P B and P+1 B should fulfll the followng equaton, { P B P F =, P+1 B P F Step 4 Transform P B P B +1 = (3) P F = P, P+1 B P+1 F = P +1 usng the prevous transformaton matrx T 1 noted as P T B. Step 5 Calculate transformaton matrx T usng K correspondences between P B K < N. and P B +1, where Step 6 Rank K control ponts accordng to the error and select K/2 control pont correspondences to calculate the transformaton matrx T. Step 7 Obtan the warp mage I T by applyng the fnal transformaton to I. Fg. 2 s the examples of mage subtractons of two regstered mages, whch show mprovement of the new method compared wth the one n our prevous study [6]. (a) Input mage (b) Prevous regstraton of mage (c) Improved regstraton of mage Fg. 2: Image regstraton effect

5 L. Zhang et al. /Journal of Computatonal Informaton Systems 10: 12 (2014) Vehcle detecton based on random walker theory Movng vehcle and statc vehcle are usually detected wth dfferent methods. In [13] and [5], movng vehcles are detected by consequental referenced mages subtracton, whle statc vehcle s not consdered. Hoogendoorn proposed a method to detect vehcle by analyzng the dfference between current mage and the background mage, whch s estmated based on the mage ntensty varaton analyss of the same poston [3]. Ths method would not be sutable for the crusng UAV, because of the dffculty of the background mage estmaton. In our prevous study, we use mage segmentaton to detect both statc and movng vehcles and use consequental referenced mage subtracton to detect movng vehcle [6]. In ths study, we adopted random walker algorthm to detect the precse vehcle boundary, whch wll mprove the accuracy of vehcle sze and poston. Random walker algorthm has been used for sem-automatc mage segmentaton n medcal mage processng [14]. It performs well for mage segmentaton, but manually labelng needed constrans ts applcaton for automatc segmentaton. In ths study, we used the subtracton result of two regstered mages as mage label for vehcle and coarse selecton area boundary as background label, whch resolves the automatc labelng problem. The experments show that ths novel algorthm could detecton the vehcle bound precsely, whch has never been reported. The algorthm s as followng: Input Two consequent mage I and I +1, where I s the th mage from vdeo and I +1 s the regstered mage of I +1 to I. Output Vehcle set detected n mage I, noted as V. Step1 Calculate the dfference mage of I and I +1. D = D (j, k), D (j, k) = { 1 I (j, k) I +1(j, k) δ 0 I (j, k) I +1(j, k) < δ (4) Where D s the dfference mage, D (j, k) s the pxel value of D at coordnate (j, k), I (j, k) and I +1(j, k) are the pxel values at poston (j, k) n mage I and I +1, and δ s the threshold value. Step 2 Create D usng morphologcal operatons on D to remove noses. Step 3 Detect blobs of vehcle canddate n I noted as B fne. Set the areas a lttle bgger than the vehcle canddate area as the coarse selecton area and note t as B coarse Step 4 Determne the seed area for background. b coarse j th vehcle n I ; bs coarse b coarse by,. (j) s the coarse selecton rectangle of the (j) s the rectangle whose wdth and heght s m pxels smaller than (j) s determned (j), whle m s set to 3 n ths paper. The background seed area b back b back (j) = b coarse (j) bs coarse (j). (5)

6 4970 L. Zhang et al. /Journal of Computatonal Informaton Systems 10: 12 (2014) Step 5 Vehcle fne detecton usng random walker algorthm. Set B fne as the vehcle area seed and b back (j) as background area seed and used the random walker algorthm as mentoned n [14] to detect the precse boundary of the vehcle noted as V movng. Step 6 Detect all vehcles V all n I usng mage segmentaton algorthms wthn road area accordng to the road surface color. Step 7 Determne all the vehcles n I. V S movng V statc = V all = {v movng v movng V movng V = V statc V movng V S movng (6) v movng V all } (7) (8) Where V S movng s the vehcles whch s movng vehcles detected n Step 5 and also detected n Step 6, and V s the vehcle set ncludng both statc vehcle and movng vehcle n I. Fg. 3(a) s one example of the vehcle boundary detecton results usng ths method, n whch the vehcles boundares ft the vehcles qute well. 3.3 Vehcle trackng method usng fast partcle fler method Vehcle trackng for arborne vdeo s a challengng problem due to frame-to-frame jtter caused by camera movement. At present, there are few attempts to deal wth multple vehcles trackng problem for arborne vdeo [15]. Partcle flter s used for vehcle trackng, whch shown to be robust. But the drawback of ths method s the hgh computatonal complexty, because the search area n the matchng process s too large. In ths paper we propose a quck partcle flter algorthm usng the searchng area constrant strategy. The man dea s to generate and move partcles only n the vehcle coarse selecton area near the predcted vehcle poston. The algorthm s as followng: Input Two consequent mage I and I +1, the poston of the tracked vehcle n I 1 and I whch s used for vehcle poston predcton n I +1 and the detected vehcle V and V +1. Output Updated vehcle trackng data set. Step 1 Predct the vehcle poston by Kalman flter, of whch the detals could be found n [17]. In the frst two frames, vehcle move speed s predefned by users. Step 2 Fnd the detected vehcle coarse selecton areas near predcted poston n I +1. Step 3 Generate partcles n selected coarse selecton areas B coarse Step 4 Move partcles wthn selected areas. defned n last secton. Step 5 Update the weght of the partcles usng hstogram matchng based algorthm. Step 6 Re-sample the partcles wthn selected areas. Fg. 3(b) shows the partcles after 3 teratons n vehcle coarse selecton areas to track vehcle 3 n Fg. 3(a). As show n Fg. 3(b), 18 partcles located n the area near vehcle 3 whch s the most, whle only 1 partcle locates n area near red vehcle whch s qute dfferent to vehcle 3 n color. Also, the best partcle marked wth small red rectangle wth label 18 n Fg. 3(b) shows that vehcle 3 s tracked correctly.

7 L. Zhang et al. /Journal of Computatonal Informaton Systems 10: 12 (2014) (a) Precse vehcle boundary detecton 4971 (b) Partcles after three teratons Fg. 3: Vehcle detecton and trackng 4 Performance Evaluaton The algorthms proposed n ths paper are mplemented and ntegrated nto the software TIEP1.0 (Traffic Informaton Extracton Platform for UAV Data), whch was developed based on the computer vson system n our prevous study [6]. TIEP s developed based on the open source GIS platform QGIS ( TIEP are mplemented usng python and C++. It can process arborne vdeo and extract vehcle trajectory n real world coordnates. The UAV used n the experment s the same one as n our prevous study [6]. Two experments for geo-statc UAV were conducted n July, To test the performance on the dense traffic hghway, Experment 1 was conducted on Caoan hghway n suburban area of Shangha, Chna. The data collected nclude arborne vdeo, UAV flght data, vehcle trajectores recorded by GPS and ODB-II frames were selected from the vdeo to test the proposed algorthm. To test the performance of the algorthm for vdeo from crusng UAV on low volume road, experment 2 was conducted on a four-lane expressway n Shnkang, Chna. The vdeo data was collected n February 2012 wth a crusng UAV, travelng about 8 klometers n ths experment. Twelve nstrumental vehcles travellng along the road were used whch are labeled wth colorfully rectangle on top of each vehcle. 500 frames were selected from the vdeo for testng purpose. Accordng to the experments, the detecton and trackng rate of the vdeo processng method presented n ths paper are 88% and 90% for experment 1 and 2, respectvely. They are hgher than the detecton and trackng rates of the method presented n [6] whch are 85% and 86% for the same data sets. 5 Summary and Concluson A vehcle detecton and trackng method s proposed to process arborne vdeo collected from UAV. To test the performance of ths method, two field experments were conducted. For the two data sets, the detecton and trackng rates of the new method are 88% and 90% whch are hgher than that of the method presented n [6], whch are 85% and 86%. The contrbutons of ths paper nclude the followng. Frst, a new computer vson system s enhanced based on our prevous study [6] n several aspects: (1) a novel accurate vehcle boundary

8 4972 L. Zhang et al. /Journal of Computatonal Informaton Systems 10: 12 (2014) detecton method based on random walker to mprove vehcle sze and poston accuracy, (2) feature pont trackng method n our prevous study has been replaced wth a quck partcle flter vehcle trackng method whch can work robustly when the feature trackng qualty s not good. The operaton cost of UAV s low and can be deployed easly and quckly. Although, processng arborne vdeo s stll a challengng problem, the rapd development of computer vson wll supply more robust and accurate algorthms. Future research s amed at mprovng the approach. References [1] Toledo T, Koutsopoulos H N, Ahmed K I, Estmaton of vehcle trajectores wth locally weghted regresson. Transportaton Research Record: Journal of the Transportaton Research Board, 2007, 1999: [2] Knoop V L, Hoogendoorn S P, van Zuylen H J. Processng traffc data collected by remote sensng [J]. Transportaton Research Record: Journal of the Transportaton Research Board, 2009, 2129: [3] Hoogendoorn S P, Van Zuylen H J, Schreuder M, et al. Mcroscopc traffc data collecton by remote sensng [J]. Transportaton Research Record: Journal of the Transportaton Research Board, 2003, 1855: [4] Cofman B, Mccord M, Mshalan R, et al. Roadway traffc montorng from an unmanned aeral vehcle [J]. IEE Journal on Intellgent Transport Systems, 2006, 1 (153): [5] Angel A, Hckman M, Mrchandan P, et al. Methods of analyzng traffc magery collected from aeral platforms [J]. IEEE Transactons on Intellgent Transportaton Systems, 2003, 4 (2): [6] Zhang L, Peng Z, Sun D J, Lu Xao-feng. A UAV-based automatc traffc ncdent detecton system for low volume roads [C]. 92rd Annual Meetng of the Transportaton Research Board, Washngton, D.C., [7] Shastry A C, Schowengerdt R A. Arborne vdeo regstraton and traffc-flow parameter estmaton [J]. IEEE Transactons on Intellgent Transportaton Systems, 2005, 6 (4): [8] Renartz P, Lachase M, Schmeer E, et al. Traffc montorng wth seral mages from arborne cameras [J]. ISPRS Journal of Photogrammetry and Remote Sensng, 2006, 61 (3): [9] Cao X, Lan J, Yan P, et al. Vehcle detecton and trackng n arborne vdeos by mult-moton layer analyss [J]. Machne Vson and Applcatons, 2010, 23 (5): [10] Cheng H, Weng C, Chen Y. Vehcle detecton n aeral survellance usng dynamc bayesan networks [J]. IEEE Transactons on Image Processng, 2011, 4 (21): [11] Pugls G, Battato S. A robust mage algnment algorthm for vdeo stablzaton purposes [J]. IEEE Transactons on Crcuts and Systems for Vdeo Technology, 2011, 21 (10): [12] Bay H, Ess A, Tuytelaars T, et al. Speeded-up robust features (SURF) [J]. Computer Vson and Image Understandng, 2008, 110 (3): [13] Luo X, Wu Y, Huang Y, et al. Vehcle flow detecton n real-tme arborne traffc survellance system [J]. Transactons of the Insttute of Measurement and Control, 2011, 33 (7): [14] Grady L. Random walks for mage segmentaton [J]. IEEE Transactons on Pattern Analyss and Machne Intellgence, 2006, 28 (11): [15] Jones R, Rstc B, Reddng N J, et al. Movng target ndcaton and trackng from movng sensors [C]. Dgtal Image Computng: Technques and Applcaton. Queensland, 2005: [16] Cao X B, Sh Z R, Yan P K, et al. Trackng vehcles as groups n arborne vdeos [J]. Neurocomputng, 2013, 99: