IMMPDA Vehicle Tracking System using Asynchronous Sensor Fusion of Radar and Vision

Transcription

1 8 IEEE Intellgent Vehles Symosum Endhoen Unersty o Tehnology Endhoen, The Netherlands, June 4-6, 8 IMMDA Vehle Trang System usng Asynhronous Sensor Fuson o Radar and Vson Feng Lu, Jan Sarbert, and Chrstoh Stller Abstrat Ths aer ouses on reognton and trang o maneuerng ehles n dense tra stuatons. We resent an asynhronous mult obstale mult sensor trang method that uses normaton rom radar and monoular son. A low leel uson method s ntegrated nto the ramewor o an IMMDA Kalman lter. Real world eerments demonstrate that the system ombnes the omlementary strengths o the emloyed sensors. A I. INTRODUCTION daned drer assstane systems or ars requre relable ereton o the ehle enronment. In artular, hgh-leel drng assstane tass le emergeny breang or ull seed range ACC neesstate a leel o relablty that may only be aheed through a ombnaton o multle sensors. Reognzng and lassyng o obets on the road as well as determnng ther oston and eloty are the ey hallenges or numerous alatons. Beng omlementary n nature, radar and monoular son may yeld obet deteton wth hgh relablty through arorate normaton uson tehnques []-[]. The nddual roertes o radar and son, as well as ther omlete otental hae been dsussed n []. Systems usng radar and son uson der manly n ther uson leel and n the synhronous or asynhronous roessng sheme. Steu and Laurgeau [] resented n ther wor a synhronous system wth low leel uson. They ombned raw data rom son and radar to rodue new raw data that are eeted to be more normate than the orgnal data. Another synhronous system ntrodued n [] uses radar targets to generate the area o nterest (AOI) n eah mage. The detetons rom these AOI are used to aldate the radar targets. In [] Sole deeloed a synhronous system wth hgh leel uson, that tras obets ndeendently by eah sensor and subsequently mathes, assoates and aldates the tras o both sensors. Ths ontrbuton rooses asynhronous roessng o radar and son data. In ontrast to the reously ted methods, the roosed aroah shall meet the ollowng requrements:. Obet anddates an be ntated rom son data as Feng Lu s wth the Drer Assstane Deartment, Robert Bosh GmbH, 79 Leonberg, Germany (e-mal: Feng.Lu@ de.bosh.om). Jan Sarbert s wth the Drer Assstane Deartment, Robert Bosh GmbH, 79 Leonberg, Germany (e-mal: Jan.Sarbert@ de.bosh.om). Chrstoh Stller s wth the Insttute or Measurement and Control Theory, Unersty o Karlsruhe, 76 Karlsruhe, Germany (e-mal: [email protected]). well as rom radar data.. The method shall wor n the eld o ew o ether sensor,.e., obets may be udated usng ether son data or radar data or both.. Obets an be lassed and aldated usng ether son data or radar data or both. 4. Obets an be traed een n hghly dynam drng maneuers. The remander o the aer s organzed as ollows: The net seton ntrodues the sensor models or radar and son as well as the eatures used or obet trang. In Seton, an nteratng multle model lter wth robablst data assoaton IMMDA s roosed. A mult sensor trang system usng monoular son and radar s ntrodued n Seton 4. Results rom eermental ehles n natural tra senes resented n Seton 5 lead to a onluson and uture wor. A. Radar II. SENSOR MODELS In our system a 77-GHz long range radar s used. It has a mamum range o m and oers an azmuth angle o,. [4]. The auray o the radar sensor s hgh n radal dreton,.e. n ts measurements o range r and range rate = r&. In angular dreton, the radar rodes oarse measurements or the lateral angle α o eah obet deteted. The radal and angular measurements are unorrelated. Thus, the measurement etor z and the measurement oarane matr R are gen as r =, α r R = ρ r ρ ρ r z, () where,, α, α r denote standard deatons and orrelaton oeent o the resete measurements. B. Camera We use a monoular amera n our system. The amera has a eld o ew (FOV) o + horzontally and +6 ertally. The mage sequene analyss algorthm searhes the mages or ossble obets ealuatng a dersty o eatures, le otal low, symmetry and shadow. Then these /8/$. 8 IEEE. 68

2 eatures are used usng the Demster-Shaer Edene Theory [5] to detet otental obet boundares. Tyally more than one boundary s generated or eah ehle. Eah boundary reresents a anddate or the lower edge o the obet,.e. the nterseton o the obet rear wth the road lane. Ths edge s sgnaled along wth ts oarane a y y z = =, R () z = w R =, (), w where z denotes the el oordnates o the obets lower enter, zdenotes obet wdth, and R,R denote the resete oarane matres. The admssble deteton range o the mage analyss algorthm s restrted to 8m, as the relablty o obet deteton sgnantly deterorates or larger dstanes. III. IMM KALMAN FILTER The nteratng multle model (IMM) Kalman lter has been ntrodued to oe wth abrutly hangng dynamal behaor [6]. Reent wor on mlementaton o IMM or real tra stuatons has been resented n [7]. The IMM lter used n ths aer aounts or tyal ehle maneuers, suh as onstant aeleraton or onstant seed rusng, and the transton between them. It s ombned wth a robablst data assoaton (DA) sheme as ntrodued n [8]. The ses o our IMMDA lter are resented n the sequel. A. Dynam models As mentoned n the ntroduton seton the ey normaton or adaned drer assstane system s the oston and the eloty o the target. An arorate hoe o the oordnate system to reresent ehle dynams and sensor normaton s rual or an arorate model: Whle measurement roertes o the emloyed sensors are best reresented n olar sensor-ed oordnates, Cartesan global oordnates are best suted to reresent ehle dynams. The adantages and dsadantages o Cartesan and olar oordnate system are dsussed n [8]. Buehren and Yang [9] resent an nterestng global oordnate model and rode a omarson between lter results or model reresentatons n sensor-ed and global oordnate systems. In ths aer a reresentaton n sem-global Cartesan oordnates s roosed. Ater the redton and nnoaton stes o the lter hae been onduted n global Cartesan oordnates, all normaton s transormed nto a oordnate system moed wth the host ehle. Two alternate dynam system models are used n our IMM lter: ) The rst dynam model assumes onstant eloty rusng: ν a a d X & = = +, (4) dt y y y y ν y ay a y where ν and ν y denote zero-mean unorrelated nose o standard deaton and y, resetely, aountng or the unertanty n aeleraton. ) The seond dynam model assumes onstant aeleraton, desrbed by: a a ν d a X & = = +, (5) dt y y y y a y a y ν ay where ν a and ν ay denote zero-mean unorrelated nose o standard deaton or the unertanty n er. B. Model nteraton a and ay, resetely, aountng The ntal state or redton o eah model s a mture o the states rom the last yle o all models wth the mng robabltes: =, =,, (6) where s the model robablty o the model n the last yle and s the robablty or the transton rom model to model. The normalzng onstants are = =,. (7) = The ntal state X and oarane o the model are: X = X = =, (8) = = [ ( X X ) ( X X ) =,, + (9) 69

3 where X and the model o the last yle. C. Model lterng s the state and oarane o Usng the ntal state and oarane the redted states and oaranes are alulated n a Kalman lter aordng to the two moton models. The nnoaton and state udate n the Kalman lter under the derent model assumtons wll be dsussed n Seton 4. D. Model robablty udate The alulaton o the lelhood unton or an IMM lter wth DA has been ntrodued n [8]. Thereore the lelhood unton Λ orresondng to the model s: Λ m = ( ) β + N( V ;, S ), () where the G D D G = D denotes the deteton robablty o a target, s the assoaton gate robablty, β s the alse alarm densty, m s the number o the assoated measurements o a ; s a Gaussan dstrbuton or the target and N (, ) nnoaton V S V wth zero mean and the oarane Thus the udated model robabltes are: S. = Λ =,, () = wth the normalzng onstant = Λ () = = E. Estmate and oarane ombnaton The mture o the model-ondtoned states estmates and oaranes yelds the resultng system state estmate and oarane aordng to the mture equatons X = X () = [ + ( X X ) ( X X ] = ) =. IV. MULTI SENSOR TRACKING SYSTEM A. Measurement transormaton (4) We use a lnear Kalman lter n Cartesan oordnates. Hene, the measurements rom radar and amera are transormed to the Cartesan oordnate model o the system. The transormaton o radar measurements s gen by z Rˆ where r = zˆ α = g R, ego g, = = r osα = / osα + y = r snα, ego (5) s the longtudnal eloty o the host ehle, and the transormaton matr g s dened as osα g = snα r snα / osα snα / os α r osα (6) In [] Sten resents a method whh omutes range and range rate usng the road geometry and the ont o ontat o the ehle and the road n mage. Aordng to ths method the longtudnal range and the lateral range y o an obet an be omuted le below: H = (7) y y =, (8) where s the ous length o the amera, H s the heght between the amera and ground, y and denote the el oordnates o the obets lower enter n () and s the el th unt. The obet wdth an be omuted as W w = (9) Due to the senstty o amera measurements on weather ondtons, ertal road urature and tlt dynams o the host ehle, range measurements o the amera may eentually be hghly naurate. Thereore, n ths aer we emloy the redted range alue rom the ntal state X o the models n (8) to alulate y and W or eah model lter. Thus the transormaton o amera oston measurements reads z Rˆ = y = h R zˆ h wth the transormaton matr = H /( y ) = y = / () 7

4 H h = y () Lewse, oordnate transormaton o the wdth s gen by z Rˆ = w zˆ = ( = W ) = w R B. robablst data assoaton / () The IMM algorthm requres that the used moton models hae dental aldaton regons n measurement sae. In ths aer, we use the ombned state redton o the system gen by = X X () where = = = X [ ( X and + ( X X oarane o the model whereas X ) ], ) (4) denotes the redted state and s the redted model robablty rom Eq. (7). Radar measurements are assoated to targets usng a aldaton gate as desrbed n [] wth a gate robablty set to.98. The assoaton o amera measurements to targets s a lttle more omle. The targets are at rst transormed to boundares n eah mage ollowng Eqs. (7), (8) and (9). Target boundares are assoated wth amera measurements they math wth the resete measurement boundary. Two seal ases are eltly onsdered: ) I a target s oluded by another target loser to the host ehle n mage, we mose that only the loser one s seen by the amera. ) In ras a radar releton may ome rom any ossble matters on the road, le ehles, trees or ola ans. In order to reent that a mong ehle may be assumed to be oluded by nearby statonary obets, we mose that mong targets may not be oluded by statonary targets. The DA roedure yelds the assoaton robabltes o the measurement wth the targets as outlned []. C. Asynhronous lterng The noton o synhrony n the ontet o mult sensor trang systems my reer to sensors or to the trang roess tsel: ) Synhrony o sensors In a synhronous sensor system all sensors tae eery measurement at the same tme nstant. In ontrast asynhronous sensors oerate ndeendently and oten een at derent measure rates. ) Synhrony o trang A synhronous trang system redts or retrodts measurements or obets that may be taen at derent tme nstanes nto seudo measurements or obets that are algned n tme []. In ontrast, asynhronous trang systems emloy eery measurement or obet uon aalablty to aldate and ntate ts tras. A man drawba o synhronous trang system aganst asynhronous system s the loss o sensor normaton when seeral measurements are omressed nto a seudo measurement. Furthermore, redton or retrodton o sensor normaton ntrodues addtonal nose to the oerall system. Last not least, the addtonal tme delay o a synhronous system may be rohbte or the untonalty o saety releant adaned drer assstane systems. For the sae o modularty, sensors tyally oerate ndeendent o another and hene rode asynhronous data. The radar and amera used n our system measure obets ndeendently. The radar has a measurement rate o Hz whle the amera oerates at 5 Hz. To ahee mamum leblty o the trang system we hae deeloed an asynhronous trang system wth low leel data uson or the asynhronous sensors. Ths allow or ull elotaton o sensor normaton and mng o sensor normaton at an early stage o the analyss roedure. Beause the roessng tme the radar sensor s sgnantly larger than the roessng tme o the amera sensor (Fg. ), the trang system reees the radar measurements ater the amera measurements een when the radar and amera sensors begn ther measurements smultaneously. Ths roblem s well dsussed n []. A theoretally otmal soluton as well as a ost-eete aromaton thereo s resented n [4]. To resole the roblem we store all reeed measurements o the amera nto a buer that s read out when new measurements o the radar arre. Then the trang system an roess the buered amera data and the ust reeed radar data sequentally n the order o ther measurement tmes. Ths roess s shown n Fg.. Fg.. The measurng tme o radar and amera 7

5 D. Target reognzng and aldaton When a target s measured by the amera we use the wdth normaton to lassy the target as a assenger ar, tru or a motoryle. Furthermore we alulate a robablty o tra estene or eah target usng the ntegrated DA (IDA) ntrodued n [5]. Fg.. The target s measured by both, the radar and the amera. The boundares det the measurements roded by the amera sensor and the trangle dets the radar deteton. The ross show the target traed by the roosed method. Fg.. Asynhronous sensor data roessng ) Radar data and amera data arre at the trang system ) All amera data MT, MT and MT amera are buered. ) The radar data MT radar and the amera data MT and MT amera, whh arred n the system between the arrng tmes o MT radar and MT radar, buld a data roessng blo that s roessed sequentally n the order o measurement by the trang system. V. RESULTS The ealuaton o the mult sensor system s based on data reorded rom drng on hghways and n urban areas. The radar measurements and amera mages were reorded together wth the longtudnal seed o the host ehle. The onstant eloty model o IMM s arameterzed as: =m/s^ and y =m/s^ The onstant aeleraton model s arameterzed as: a =m/s^ and ay =5m/s^ The transton matr o the IMM lter s set to.98. M (.s) =...98 The ntaton alues o the models are. or the CV model and.8 or the CA model. Fg. 4. The target s turnng let and s only measured by the amera. Fgs. and 4 show results rom a tyal urban drng sene where a ehle s ollowed by the host ehle. Fgs. 5-8 show the traed arameters together wth the measurements. Usng the normaton o the amera sensor the target s also orretly lassed as assenger ar wth a wdth o.55 meter. VI. CONCLUSIONS In ths aer we resent an IMMDA mult sensor trang system usng asynhronous roessng o measurements o radar and son. The result o the smulaton usng measurements n real tra stuatons shows that the trang system ombnes the adantages o both sensors and the IMM wors orretly. In Fg. 8 we an see that the range measurements o amera are unrelable oer a dstane o 5 meter due to th dynams o the host ehle. Een though the roosed system oes well wth ths stuaton, uture wor on mage stablzaton ould ontrbuton to dretly mroe the measurement data. In artular, ths wll enhane the trang erormane or targets that are only measured by the amera. 7

6 Fg. 5. Traed range o the target n [m]. Due to ts hgh auray the traed range s domnated by the range measurements o the radar. Fg. 8. Angle n [ ] omuted rom the traed lateral dstane range. Due to ts hgh auray the traed angle s domnated by the amera measurements. Fg. 6. Traed eloty o the target n [m/s]. Only the radar an measure the eloty nstantaneously. At the.5 s and s the target braes strongly. Between. s and.5 s and rom s onwards the target ruses at almost onstant eloty. Fg. 7. Model robablty o the onstant eloty model. At the s and s the IMM lter swthes rom onstant eloty model to onstant aeleraton model. REFERENCES [] B. Steu, C. Laurgeau, L. Salesse, and D. Wauter, Fade: A ehle deteton and trang system eaturng monoular olor son and radar data uson, n ro. IEEE Intell. Vehles Sym., Versalles, Frane, Jun., [] G. Alessandrett, A. Brogg, and. Cerr, Vehle and Guard Ral Deteton Usng Radar and Vson Data Fuson, IEEE Trans. Intell. Trans. Systems, Vol. 8, No., Marh 7. [] A. Sole, O. Mano, G.. Sten, H. Kumon, Y. Tamatsu and A. Shashua, Sold or not sold: Vson or radar target aldaton, n ro. IEEE Intell. Vehles Sym., arma, Italy, Jun. 4, [4] Robert Bosh GmbH and H. Bauer, Adate Fahrgeshwndgetsregelung ACC, Chrsan, Konstanz,. [5] S. Smon, W. Nehsen, A. Klotz and B. Luas, Vdeo-baserte Obet-Deteton und Veraton on Radar-Obet-Hyothesen ür Komort- und Sherhetsuntonen, n Worsho Fahrerassstenzsysteme FAS 5, Germany, Ar. 5. [6] H. A.. Blom, An eent lter or abrutly hangng systems. In ro. IEEE Con. on Deson and Control, , 984. [7] N. Kaemhen, K. Wess, M. Shaeer and K. C. J. Detmayer, IMM obet trang or hgh dynam drng maneuers, n IEEE Intell. Vehles Sym., arma, Italy, Jun. 4, [8] S. Blaman and R. ool, Desgn and Analyss o Modern Trang Systems, arteh house ed., Norwood, MA,. [9] M. Buehren and B. Yang, A Global Moton Model For Target Trang n Automote Alatons, n ro. IEEE ntl. Con. On Aousts, Seeh and Sgnal, Hawa, Ar. 7, Vol., [] G.. Sten, O. Mano, and A. Shashua, Vson-based ACC wth a sngle amera: bounds on range and range rate auray, n ro. IEEE Intell. Vehles Sym., Columbus, Oho, Jun., [] Y. B. Shalom and T. E. Fortmann, Trang and Data Assoaton, Aadem ress INC., Orlando, 988. [] X. Ln, Y. B. Shalom and T. Krubaraan, Multsensor-Multtarget Bas Estmaton or General Asynhronous Sensors, IEEE Trans. On Aerosae and eletron systems, Vol. 4, No., Jul. 5. [] N. Kaemhen and K. C. J. Detmayer, Data synhronzaton strateges or mult-sensor uson, n ro. o ITS, th World Congress on Intellgent Transortaton Systems, Madrd, San, No.. [4] Y. Bar-Shalom, H. Chen, and M. Mall, One-ste soluton to the mult-ste out-o-sequene measurement roblem n trang, IEEE Trans. Aerosae and Eletron Systems, ol. 4,. 7-7, 4. [5] D. Mus, R. Eans and S. Stano, Integrated robablst Data Assoaton, IEEE Trans. On Automat Control, Vol. 9, No. 6, Jun