A New Technique for Vehicle Tracking on the Assumption of Stratospheric Platforms. Department of Civil Engineering, University of Tokyo **

Transcription

1 Fuse, Taash A New Technque for Vehcle Tracng on the Assumton of Stratosherc Platforms Taash FUSE * and Ehan SHIMIZU ** * Deartment of Cvl Engneerng, Unversty of Toyo ** Professor, Deartment of Cvl Engneerng, Unversty of Toyo Hongo 7-3-1, Bunyo-u, Toyo, E-mal: <fuse, shmzu>@lanner.t.u-toyo.ac. JAPAN KEY WORDS: Obect Tracng, Movement Detecton, Hgh Resoluton Data/Images, Image Sequence, Platforms, Algorthms, Stochastc. ABSTRACT Traffc flow survey for traffc control and lannng s usually conducted wth traffc beacons set u at lmted roadsde onts. Therefore, they cannot observe the exact dynamc movement of vehcles whch s mortant nformaton for sohstcated traffc olcy. On the other hand, stratosherc latform system has been recently roected n Jaan. One of the uroses of the stratosherc latform system s utlzaton for earth observaton. The stratosherc latform s exected to result n hgh satal and tme resoluton mages at secfc areas for contnuous observatons. These hgh resoluton and contnuous mages certanly mae observaton of vehcle movement easer. In ths aer, we exlored the ossblty of vehcle tracng wth hgh resoluton and tme-seral aeral mages, whch are on the assumton of the use of stratosherc latforms. In estmaton of dslacement vectors of vehcles, the most characterstc roblem s that aearance/dsaearance of vehcles occur, when they are under overhead brdges or shadows of buldngs, or gong outsde the mage, or so on. We emloyed the robablstc relaxaton method for tracng vehcles. And then we mroved the robablstc relaxaton method by ntroducng (1) the color nformaton of vehcles, and () the dslacement vectors of each other. We aled the roosed method to smulated data and samle mages, whch were on a one-way street. The tme nterval of successve mages was 1.5 seconds. The roosed method yelded a better result than the orgnal method, and the rate of correct corresondence s above 95%. Furthermore, we also aled to the varous tme nterval mages. When tme nterval was less than 1.5 seconds, the result was good for vehcle tracng n ths case. 1. INTRODUCTION Traffc flow survey for traffc control and lannng s usually conducted wth traffc beacons set u at lmted roadsde onts. Therefore, they cannot observe the exact dynamc movement of each vehcle whch s mortant nformaton for olces decson for traffc roblems. There have been a few attemts made to study the survey of dynamc movement of vehcles, for examle, wth aeral hotograhs. The results, however, were not suffcent for useful alcaton to traffc engneerng. In recent years, a stratosherc latform system has been roected manly by the Scence and Technology Agency and the Mnstry of Posts and Telecommuncatons n Jaan. It s bascally ntended to contrbute to telecommuncaton and the other dfferent uroses. One of them s utlzaton for earth observaton. The stratosherc latform s beng suosed to be et at a stratosherc alttude of about 0m, so t s exected to result n hgh satal and tme resoluton mages at secfc areas for contnuous observatons. These hgh resoluton and contnuous mages certanly mae observaton of vehcle movement easer. As a result, the stratosherc latform has a great otental wth wder scoe of utlzaton, for nstance, survey for tang countermeasure aganst traffc am, for tang orgn-destnaton data, survey of rate of rght and left turn n ntersectons for sgnal control, and so on. In ths aer, we exlore the ossblty of vehcle tracng wth hgh resoluton and tme-seral aeral mages, whch are on the assumton of the use of stratosherc latforms. To be secfc, we develo a new technque for vehcle tracng wth hgh satal and tme resoluton remotely sensed mages. We roose mroved robablstc relaxaton method as the new technque by ntroducng (1) the color nformaton of vehcles, and () the dslacement vectors of each other. And then, we confrm the effectveness of the roosed method through alcatons to smulated data, samle mages and dfferent tme nterval successve mages. Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam

2 Fuse, Taash. VEHICLE TRACKING PROBLEM The vehcle tracng roblem bascally conssts of two comonents. The frst s how to detect vehcles automatcally n each mage. The next s how to estmate dslacement vectors of vehcles n successve mages. Though sensors on the stratosherc latforms are not decded yet, tang account of alttude of about 0m, t s exected to result n hgh satal resoluton mages. We can farly exect the satal resoluton to be n the range of 0cm-50cm. Fgure 1 comares between a hgh resoluton mage (a), whch exected to be roduced by stratosherc latforms, and a usual satellte s mage (b). Hgh resoluton mage (a) has resoluton of 30cm and usual satellte s mage (b) has resoluton of 30m. It s obvous that vehcles can be recognzed easly n the hgh resoluton mage. Usng such hgh resoluton mages, t s exected to be comaratvely easer to detect vehcles accurately. In ths aer, we were concerned wth the second comonent of the roblem, that s estmaton of dslacement vectors. (a) Hgh Resoluton Image (Aeral HDTV Image, 30cm) (b) Usual Satellte s Image (LANDSAT TM, 30m) Fgure 1: Comarson of Hgh Resoluton Image wth Ordnary Satellte s Image. The estmaton of dslacement vector secfes the orgns and destnatons of all vehcles n successve mages. When the vehcles are detected as n Fgure, the vehcle A n the mage 1 can move to C, D or E n the mage or dsaear. The most characterstc roblem for the vehcle tracng s that aearance/dsaearance of vehcles occur, when they are under overhead brdges or shadows of buldngs, or gong outsde the mage, or so on. The exstence of the aearance/dsaearance mae the tas of vehcle tracng challengng. Image 1 A B t C D Image E Fgure : Vehcle Tracng Problem. 3. PROBABILISTIC RELAXATION METHOD AND IMPROVEMENT OF THE METHOD The relaxaton method was orgnally develoed as an algorthm for numercal calculaton (Taag and Shmoda, 1991). The relaxaton method has been wdely emloyed for mage matchng technques (Zucer, Hummel and Rosenfeld, 1977, Barnard and Thomson, 1980, Peleg, 1980, Ohm and Yu, 1998, Saamoto, Uchda and Wang, 1998). In the mage matchng, sets of canddate matchng onts are frst selected ndeendently n each mage. An ntal networ of ossble matches between the two sets of canddates s then constructed. An ntal estmate of the robablty of each ossble match s made equally. Fnally, these estmates are teratvely mroved by a relaxaton labelng technque mang use of the local consstency roerty of dslacement vectors, that s smlarty between the dslacement vectors of near canddate onts. We emloyed the robablstc relaxaton method for tracng vehcles, because the smlarty between the dslacement vectors s arorate to a characterstc of movement of vehcles. The robablstc relaxaton method s summarzed 78 Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam 000.

3 Fuse, Taash as follows n alyng to vehcle tracng (Barnard and Thomson, 1980, Taag and Shmoda, 1991). We used two successve mages, Image 1 and Image. At frst, we select canddates n Image corresond to a vehcle n Image 1. The canddates are restrcted by the velocty of the vehcle. These are referred to as ossble corresondences. They nclude no corresondence due to the dsaearance of the vehcle. And then, robabltes to the corresondences are set equally. The robabltes are mroved successvely by alyng a consstency roerty, that s the smlarty between dslacement vectors of near vehcles. Fnally, the most ossble corresondence wll have the hghest robablty. However, the robablstc relaxaton method cannot tae the color of vehcles nto account, whch s very sgnfcant nformaton, and t cannot be aled to the aearance of vehcles. Tang countermeasure aganst these roblems, we mrove the robablstc relaxaton method by ntroducng: (a) the color nformaton of vehcles, and (b) the dslacement vectors of each other. (1) Secfyng Intal Probabltes Color nformaton s very sgnfcant for corresondence between vehcles. So the color nformaton s ntroduced to secfyng ntal robabltes of the ossble corresondences, namely the ntal robabltes of corresondences are vared accordng to smlarty of the colors. From now, we ntroduce abbrevatons as follows: r 1 (x, y), g 1 (x, y), b 1 (x, y): Intensty of R, G, B (8bt) n Image 1, r (x, y), g (x, y), b (x, y): Intensty of R, G, B (8bt) n Image. In both mages, the value of ntensty s D: Maxmum of vehcles movement (xels). a : Vehcles detected n Image 1 locate (x, y ). =1,,, m. b : Vehcles detected n Image locate (x, y ). =1,,, n. : Label of dslacement vectors. 0 r, g, b 55 = C. (1) max ={ 1,,, n, } () : Label of no corresondence. λ = ( x, y ), D x D, D y D ( ) (3) P ( ): Probablty that vehcle a has label. We refer to label robablty. P ( λ ) = 1, 0 P ( λ ) 1. (4) If b (l =1,,, L) are canddates corresondng to a, a has L+1 label. L ={ 1,,, L, }, (5) where λ λ λ = ( x x, y y ) = ( x, y ). (6), The square of dstance of vehcles color n the RGB sace s C ( )=(r 1 (x,y )-r (x,y )) +(g 1 (x,y )-g (x,y )) +(b 1 (x,y )-b (x,y )). (7) Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam

4 Fuse, Taash At the no corresondence, the dstance of color s defned by exectaton of dsarty of the color Let and then, the ntal label robablty s 1 C ( )= 3 C max. (8) 3 ) C ( ) C ( λ λ l ( λ ) =, (9) C ( λ ) (0) At the no corresondence, the ntal label robablty s ()Imrovement of Label Probablty P (0) P l l ( λ ) ( λ ) =. (10) ( λ ) l ( λ ) ( λ ) =. (11) ( λ ) In the rocess of mrovng robabltes, the dslacement vectors from Image 1 to Image and from Image to Image 1 are used. We refer the dslacement vectors from Image to Image 1 to ooste dslacement vectors. Ths method can be aled to the aearance of vehcles usng the dslacement vectors of each other. We also ntroduce the addtonal abbrevatons as follows: : Label for ooste dslacement vectors. : Label of no corresondence, and ={ 1,,, m, }. (1) λ ' = ( x, y ), D x D, D y D ( ). (13) Q ( ): Probablty that vehcle b has label. We refer to ooste label robablty. Q ( λ ) = 1, 0 Q ( λ ) 1 (14) A vehcle a locates (x, y ) near a vehcle a. When vehcle a, whch has hgh label robablty P ( ), exsts near vehcle a, t s consstent that vehcle a has label. Ths roerty s called as consstency roerty. In ths case, the label robablty s ncreased. Degree of consstency are defned as follows P ˆ ( λ ) = P ( λ ), (l=1,,,l), (15) l L Q ˆ ( λ ) = Q ( λ ), (l=1,,,l). (16) l L 80 Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam 000.

5 Fuse, Taash Labels are consdered to be consstent f they reresent nearly the same dslacements, ( x x ) + ( y y ) T. (17) L n the equatons (15) and (16) s a set of labels whch satsfy equaton (17). When vehcle a does not have the label whch satsfy equaton (17), degree of consstency s Pˆ ( λ ) =0, otherwse Pˆ ( λ ) >0. Vehcle a are selected by satsfyng equaton (18). ( x x ) + ( y y ) R. (18) For ooste label, degree of consstency s defned by (16). Imrovement of label robabltes s accomlshed by alyng followng equaton, where and For ooste label robabltes, where and Q' P P ( λ ) ( λ ) = (l=1,,, L, ), (19) P ( λ ) l L P ( λ ) = P ( λ ), (0) ˆ P ( λ ) = P ( λ )( A + BP ( λ ) + CQ ( λ )), (l=1,,,l). (1) Q l L Q' ( λ' ) ( λ ' ) =, (l=1,,, L, ), () Q ( λ' ) Q' ( λ ' ) = Q ( λ' ), (3) ˆ ( λ ' ) = Q ( λ' )( A + BQ ( λ' ) + CP ( λ' )), (l=1,,,l). (4) Parameters A, B and C n equaton (1) and (4) are ostve constants whch nfluence the convergence characterstcs of the model. The role of A s to delay the total suresson of unlely labels. The role of B and C s to determne the rate of convergence. The larger the value of B and C relatve to A, the faster wll be the convergence of smlarty assgnments. The comlete rocedure to estmate the most lely corresondence for each vehcle can be summarzed as follows. Each ossble corresondence s assgned an ntal robablty. These robabltes are teratvely refned usng equaton from (19) to (4). Ths rocedure s reeated untl the robabltes reach steady states, but n ractce we may need to arbtrarly sto t at 50 teratons (Barnard and Thomson, 1980, Taag and Shmoda, 1991). Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam

6 Fuse, Taash 4. EXPERIMENTS 4.1 Confrmaton of Effectveness of Proosed Method We aled the orgnal robablstc relaxaton method and the roosed method to smulated data. We emloyed traffc mcro smulator PARAMICS (PARAMICS TRAFIIC SIMULATION LTD.) for roducng the smulated data. The data were on a one-way and two-lane road. The tme nterval of successve mages was 1.5 seconds. Detected vehcles are 5 n Image 1 and 51 (one vehcle dsaeared) n Image, resectvely. We aled the followng method: (a) Orgnal robablstc relaxaton method (orgnal method); The ntal robabltes are secfed equally, and only one drecton label robabltes are consdered. (b) Probablstc relaxaton method wth color nformaton (wth color nformaton); The ntal robabltes are secfed by tang account of color nformaton, and only one drecton label robabltes are consdered. (c) Probablstc relaxaton method wth ooste label (wth ooste label); The ntal robabltes are secfed equally, and two drecton label robabltes are consdered. (d) Proosed method; The ntal robabltes are secfed by tang account of color nformaton, and two drecton label robabltes are consdered. Parameter A, B and C n (1) and (4) were secfed based on results. We aled these methods to two tyes of mages. In the frst, all vehcles were detected ( (1) wthout aearance and dsaearance: Wthout), n second, 10% of vehcles were not detected ntentonally ( () wth aearance and dsaearance: Wth). Table 1 shows the results. Table 1: Correct Rates to Smulated Data. (1) Wthout () Wth (a) Orgnal method 8.7% 80.1% (b) Wth color nformaton 88.5% 91.5% (c) Wth ooste label 98.1% 97.8% (d) Proosed method 100.0% 100.0% The correct rate by the roosed method are 100.0% n the both cases, that are wthout aearance/dsaearance, and wth aearance/dsaearance. These results verfy the effectveness of the roosed method and robustness to aearance/dsaearance roblem. 4. Alcaton to Samle Images 4..1 Secfcaton of Parameters wth Smulated Data When the roosed method are aled to real mage, some arameters must be secfed. Parameters T, R and D (T: threshold for consstency, R: threshold for neghborhood, D: maxmum of vehcles movement) can be secfed easly by consderng the lmt of vehcle velocty n the road of nterest, and so on. Parameters A, B and C n (1) and (4), however, cannot be secfed easly. Because these arameters are affected by state of traffc flow, tme nterval of successve mages, and so on. It s arorate that these arameters are secfed by alyng to smulated data whch assume the real traffc flow. To be secfc, smulated data reroduce the real traffc flow of nterest. The methods are aled to smulated data wth the varous value of arameters, and then the results are comared. Based on the comarson, the arameters wll be secfed. In ths aer, we used followng value of arameters. A= 0.5, B= 1, C= 1, T=0, R=00, D =150. Here, we vared value of A from 0 to 1 at an nterval of 0.1, values of B and C from 0 to 10 at an nterval Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam 000.

7 Fuse, Taash 4.. Alcaton to a Samle Image We also aled the above-mentoned four methods to a samle mage roduced from an aeral HDTV mage (Fgure 3). The data of ths mage are as follows: Platform: Helcoter; Tme: 1996, March 16th; Area: Yodogawa-Rver Area; Length of the road: about 700m (one-way and two-lane); Alttude of latform: about 500m; Tme nterval of successve mage: 1.5 seconds; Satal resoluton: 0.33m ; Number of channels: 3 (R, G and B, 8bt); Sze of mage: 190 by 650 xels ; Number of detected vehcles: 47 n Image 1, 48 n Image (one vehcle aeared). Fgure 3: Samle Image. We used the value of arameters secfed n the Secton We aled the four methods to both of tyes of mages as mentoned n Secton 4.1. Vehcle detecton was accomlshed manually. The coordnates of vehcles were the center of gravty, and the colors were the average of the each vehcle. For calculatng the correct rate, accurate labelng were constructed wth successve mages at a short nterval manually. Table shows the results wth the samle mage. The roosed method roduced mroved results than the orgnal method, and the rate of correct corresondence s above 95%. In the case of (c) wth ooste label, the results were better than (a) orgnal method and (b) wth color nformaton. However, they were much worse than those by the roosed method, because there exsted vehcles whose colors were not smlar and dslacement vectors were smlar. The algorthm of relaxaton method mroves the robabltes usng only consstency roerty. So, f these vehcles exst, the label robabltes cannot reach convergence. The roosed method has advantage over (c) wth ooste label method due to use color nformaton. Table : Correct Rates to Samle Image. (1) Wthout () Wth (a) Orgnal method 78.7% 75.6% (b) Wth color nformaton 80.9% 75.6% (c) Wth ooste label 83.% 78.9% (d) Proosed method 95.8% 96.7% Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam

8 Fuse, Taash 4.3 Alcaton to Dfferent Tme Interval Successve Image Furthermore, we also aled the roosed method to some successve mages at dfferent tme ntervals, whch were 0.5s, 1.5s,.5s and 4.0s. The data of the mages are same as those n Secton 4.. Table 3 shows the results. When the tme ntervals were less than 1.5 seconds, the correct rates were good. On the contrary, when the tme ntervals were over.5 seconds, the correct rates were less than 80%. When the tme nterval s less than 1.5 seconds, the accuracy of vehcle tracng s suffcent n ths case. It s note that the relaton between tme nterval and accuracy s much affected by states of traffc flow. Table 3: Correct Rates to Dfferent Tme Interval Images. t (seconds) Correct rates % % % % For all the methods mentoned above (Secton 4.1 to 4.3), the rocessng tme s about 60 seconds wth a usual comuter (CPU: Pentum 66MHz, RAM: 18MB). 5. CONCLUSION The conclusons of ths aer are as follows: (1) We roosed a new method by mrovng robablstc relaxaton method; () We confrmed the effectveness of the roosed method through alcatons to smulated data, samle mages and some dfferent tme nterval successve mages. The future wors are as follows: (1) Alcaton to more comlex traffc flow; () Comarson wth other matchng method; () Develoment of automatc vehcle detecton method; (3) Unfcaton of detecton method and tracng method as vehcle tracng system. REFERENCES Barnard, S.T. and Thomson, W.B., Dsarty Analyss of Images. IEEE Transacton of Pattern Analyss and Machne Intellgence, PAMI- (4), Ohm, K. and Yu, L.H., Performance of the Relaxaton Method PTV on the Bass of PIV Standard Images. Journal of Vsualzaton Socety of Jaan, 18 (), Peleg, S., A New Probablstc Relaxaton Scheme. IEEE Transactons of Pattern Analyss and Machne Intellgence, PAMI- (4), Saamoto, M., Uchda, O. and Wang, P., Automatc Hgh Accuracy Te Ponts Detecton n Stereo Images Usng Relaxaton Method and Gradent Method. Journal of the Jaan Socety of Photogrammetry and Remote Sensng, 37 (5), Taag, M. and Shmoda, H., Handboo of Image Analyss. The Unversty of Toyo, Toyo, Zucer, S.W., Hummel, R.A. and Rosenfeld, A., An Alcaton of Relaxaton Labelng to Lne and Curve Enhancement. IEEE Transactons of Comut., C-6 (4), Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B5. Amsterdam 000.