A Mult-mode Image Tracng System Based on Dstrbuted Fuson Ln zheng Chongzhao Han Dongguang Zuo Hongsen Yan School of Electroncs & nformaton engneerng, X an Jaotong Unversty X an, Shaanx, Chna Lnzheng@malst.xjtu.edu.cn czhan@xjtu.edu.cn zdg_ll_zly@63.net Abstract In ths paper, a mult-mode mage tracng system was desgned, whch use the dstrbuted structure to fuse the tracng results of these modes. Because of the nose n mage, the presence of occluson, and the deformaton of traced target, some tracng algorthm may fal. So t s the man dea n ths paper to merge the advantages from all of the tracng algorthms, and then have a robust behave. There are three tracng models appled n ths syste whch are the modes based on regon, based on B-snae, and based on dstngushed ponts. Each of these three modes contrbutes to the fuson center, and communcates to other models. The experence results show that ths system could overcome the common problems n tracng. Keywords: Image tracng, mult-mode, correlaton coeffcent, fuson Introducton Image tracng s an mportant researchng feld n mage processng and computer vson. In the tracng process, a lot of message related to the movng targets have been lost, because of the 3D to D projecton. As a result, mage tracng s normally an uncertan proble and some pror nowledge s appled to ths process to solve t. There are many mage tracng algorthms proposed now [][][3][4], whch utlze dfferent mage nformaton, and have dfference presuppostons. Each mode has ts own advantages and dsadvantages, and sometmes they can complment each other. So t s reasonable to fuse these tracng methods, and tae advantage of all of them. There are three tracng modes used n ths system. The frst s based on regon. Ths mode s stable f there s no serous occluson, and could locate the target accurately. Its man dsadvantages nclude two aspects. Under sunlght, the shadow of target may change, and sometmes tends to devod of target s texture. So mae the tracng dffcult. Secondly, f the target s occluded by another target, t s usually dffcult to dstngush these wth the method based on matchng. The second tracng mode s based on actve contour. The man characters of ths method are ts rapd convergng and accurate segment. But t has two defects. Frstly, the actve contour needs to be posted near the real contour before convergng, but an unease thng to do. Secondly, t may be wrong when there s even slght occluson. The last tracng mode s based on dstngushed ponts, whch are the salent character ponts on the target. The advantage of ths approach s that even n the presence of partal occluson, some of these ponts reman vsble and could be traced. The man dsadvantage of ths mode s that, the calculatng s very burdensome f the number of traced ponts s huge. And ths nd tracng method s less accurate than the prevous two modes. There are two basc structure used for fuse these tracng modes. One s centralzed structure, and the other s dstrbuton structure. Wthn the centralzed syste every sensor sends the orgn measurements to fuson center, so t s burdensome n the center and n the networ. Contrastvely, wthn the dstrbuton syste every sensor has ts own predctor, and just sends ths to the center. It s not a hard wor n fuson center wth dstrbuton structure, and the system s balanced. Thus n ths paper, dstrbuted structure s accepted. In fuson center, a Kalman flter was used. By estmatng the poston of the target s center n the fuson center, Matchng mass of regon-based tracng and dstngushed ponts based tracng are posted at the coarse postons respectvely. And the actve contour obtaned n last tracng step s also posted correctly n current step. Then wth searchng methods, these modes fnd out the correspondng measurements and update the condton vector, the covarance matrx respectvely. After that, these data are sent to the fuson center, whch wll update the condton vector and covarance matrx globally. As a result, advantages of each tracng mode are merged, whch maybe more accurate, and stable than a sngle tracng mode. In ths syste although the same mage sequence s analyzed, each tracng mode could get dfference part of nformaton from orgnal mages. By the way, t s possble for them to exchange nformaton each other. Snce the regon based tracng has used whole nformaton n the mage, t plays a ey role among these three tracng mode. Durng the tracng process based on regon, correlaton coeffcent between the mas and mage can be obtaned. Wth the coeffcent we can decde whether the target has been occluded, and the degree of occluson. The correlaton coeffcent s used as the belef measure for occluson. By t, the regon based tracng mode decdes f the tracng results of other two modes 83
wll be sent to fuson center. On the other hand, the actve contour method also plays an mportant role to the other modes, such as supplyng the ntal or updated mass for regon-based and dstngushed ponts based tracng. Ths paper s organzed as followng. In secton, these three tracng modes are constructed respectvely. The dstrbuted fuson system s bult n secton 3, whch conssts of a alman flter and these three modes. In secton 4, several experences are done wth ths system. And secton 5 s the concluson. Desgn of tracng modes In the mult-mode tracng syste three tracng modes are used, whch are the modes based on regon, based on actve contour, and based on dstngushed ponts. These three modes have advantanges and dsadvantages respectvely, and could complement each other.. Tracng mode based on actve contour In mage segment or mage tracng, actve contour s becomng a useful tool, whch s often a close curve, and deformable[5]. In an energy feld that correpondng to the edge nformaton n mage, the actve contour could graduately approach the real contour of target untl reach t. The man advantage of t than tradtonal segment method s that even n the presence of dscontnue or blurry edges, the actve contour segment the target well. An actve contour can be represented by a number of control ponts, whch determne the actve contour s shape. In ths paper, B-splne actve contour s used, whch s very smooth and the control ponts number of t s very lttle [6]. For a presented target, the number of control ponts also decde the resoluton of segment. In mage tracng, a coarse contour s adequate. So lttle number of control ponts are used, and most detal parts along the real contour have been neglected. A correct segment wth actve contour n the frst mage s the essence for the next tracng of t and other two tracng modes. In the frst mage, f the result actve contour devate from real contour, t wll devate from target n followng mage sequence. Snce the result actve contour s also used to determne mass of regon based tracng and dstngushed ponted based tracng, a wrong actve contour may deform these mass, and decrease the matchng accurate of these two tracng modes. In ths paper, Accumulatve dfference method s used for coarsely segmentng the targets that enter the detecton regon of mage. Then the edge of the coarse segment s used as the ntal poston for actve contour. Pushed by the mage force, actve contour wll gradually converge to a stable poston whch compactly envelops the target n the frst mage. As a result, the segment of target s also obtaned. For the reason that an actve contour s deformable, t s used to trac the movng target, whose movement s not rapd. But f the target move very rapd and so exceed the trappng range of energy feld, the snae may lose t. For ths reason, t s neccesary to use a estmator for predct the coarse poston n the next mage. If the maneuverablty of targets s not strong, a common Kalman flter s sutable for the stuaton. Experences show that, the combne of actve contour and Kalman flter porform well n tracng[7]. Most actve contour has the ablty to shrn. It s reasonable for us to let the ntal actve contour beyond the target s real boundary n every mage. The detaled method s that, let the actve contour obtaned from prevous mage located at the predcted poston n current mage, then enlarge t by the scale.04~.08. Then the snae wll move stably, and the segment qualty can be mproved. The man advantages of actve contour method are ts hgh segment accurate, less expensve of tme.. Tracng mode based on regon Tracng mode based on regon s a mas matchng process. By analyzng the correlaton coeffcents between the mas and a search wndow wthn mage, the target s center s determned n the mage sequence []. It provdes an accurate locaton for target because t uses the full nformaton of target, such as color, texture, etc. There are two man factors nfluence the behavor of ths tracng mode. Frstly, the mas couldn t be posted too far from the target s true poston. Otherwse the searchng wndow needs to be very large for tracng t, and the tme expense also ncreases. For solvng ths proble Kalman flter same as prevous subsecton can be used. Secondly, the mas must reflect the change of target. Usng the segment result of actve contour, the mas could be obtaned and updated f necessary. The mas plays an mportant role n tracng. For gettng t, segment result of actve contour s used. By enclosng the segment wth a rectangle, whose edges are tangent to the segment, and rest parts are full wth zeros, the mas can be got. Once the mas s set to the predcted poston accordng to the Kalman flter s estmaton, a searchng wor for best match starts. If the correlaton coeffcent s defned as C( R( = C( C ( where, M N C ( = f(, j) MN = j= and C ( M ) ( N ) ( = f ( m +, n + j) MN = ( M ) j= ( N ) 833
C( MN M N = = j= f (, j) M + N + f ( m +, n + j ), and f(, j) s the mas, whose sze s MN. f s the current mage. ( s a searchng poston n the searchng wndow, whch s a rectangle area n f and centralzes the predcted poston. Then the poston correspondng to the maxmum correlaton coeffcent wthn the searchng wndow s determned as the measurement for target center. Through the tracng, f the target manly moves n translaton mode, the projecton of target may not change except the scale. So the scale of mas must change automatcally. To do ths wor, two new mass are used, whch ncreases, decreases the scale respectvely. The mas that provdes the best matchng s selected and preserved for next tracng step..3 Tracng based on dstngushed ponts Tracng mode based on dstngushed ponts s also usng mas matchng. But the mass are small regons of the target, whch centralze dstngushed ponts [8]. Dstngushed ponts are the salent ponts on the target. Fndng them out s the ey n ths tracng mode. There are many methods to do ths wor. In ths paper, a method based on quadrature par of flters s appled to analyss these ponts [9]. A smple form of quadrature par of flters s the combnaton of the steerable flter and ts Hlbert transfor wth whch the local orented energy and the correspondng domnant orent can be obtan. In ths paper, the second dervatve of the Gaussan functon along x drecton, s selected as the steerable flter: ( x + y ) ( x + y ) G = e = (4x ) e x ts Hlbert transform denote as H, then the energy along any drecton θ s θ E( x, y, θ ) = I( x, * G () θ + I( x, * H Accordng to method n [0], the domnant orentaton θ d and local orented energy E d are calculated, whch correspond to the maxmum orented energy at every pxel. E d reflects the texture or edge map n the mage. Usng the method descrbed n [9], frstly, we transform the domnant orentaton n θ d at every 0 pxel wth 90, and get the vertcal orentaton map θ. Then the equaton () s used to calculate the v orentaton energy n map E d along orentaton θ v at every pxel respectvely. So the result-energy map E v s obtaned, from whch the dstngushed ponts can be found out. Referrng to the segment of target wth actve contour method, a dstngushed ponts detecton algorthm s proposed. In map E v, ponts that satsfy the followng three condtons may be dstngushed ponts: () These ponts are wthn the range of target s segment. () In map E v, the energy of these pont s hgher than a threshold T = (0.8 ~ )E average, where E average s the average energy n E v. (3) In map E v, the energy of these ponts s local maxmum wthn a regon, whch sze s about ~ of the total area of mage. 6 5 As a result, there may be 4~6 dstngushng ponts obtaned. Every dstngushng pont has ts own mas for tracng usng matchng method. But they are dfferent from the mas used n regon based tracng mode, because they are just some small local areas on the target. Snce these dstngushng ponts dstrbute wdely on the target, there may be some of them could be seen even when the target s serously occluded. So the man advantage of ths nd of tracng mode s that t behaves well when occluson exts. The man dsadvantages of ths method nclude two aspects. Frstly, t s tme expensve because there are too many pont need to be matched. Secondly, the accurate s not hgh. Snce the mas s small, the matchng may be nfluence by nose or other smlar structure on target. 3 The dstrbuted fuson system In ths paper, a dstrbuted fuson system s used to merge the tracng results from three tracng modes. Snce the data t receves s very smple, t wll wor easly and the data exchange n networ s not burdensome []. See fgure, the system conssts of a fuson center and three tracng models. The fuson center s a specal Kalman flter, whch feedbacs to every tracng mode, and thus unfes the condton vector and covarance n the three sub-tracng modes. In ths paper, the three tracng mode has the common condton vector, whch conssts of fve elements. T x ˆ = [ x y x& y& λ] where, ( x, y ) s the coordnaton of target s center. And ( x &, y& ) s the velocty of target s center. λ s the scale f the projecton of target enlarges or shrns. 834
Wth the same estmate of condton vector xˆ( t, t ) = [ x( t t ) y( t t ) x& ( t t ) T y& ( t t ) λ( t t )], whch obtaned by dstrbuted fuson center, every tracng mode would fnd the correspondng coarse poston n tme t. For the tracng modes based on actve contour and regon, the ntal actve contour and mas are just place at ( x ( t t ), y( t t )). For the tracng based on dstngushed ponts, the center of -th mas has a devaton from ( x ( t t ), y( t t )) accordng to scale λ. x = x( t t ) + λ( t t ) x y = y( t, t ) + λ( t t ) * y, =,,L N where, ( x, y ) s the devaton of -th dstngushng pont from target s center n the frst mage. N s the number of dstngushng ponts. Wth these measurements, every tracng mode updates ts own observaton vector and covarance matrx wth customary Kalman equaton []. Then these data are translated to fuson center. Accordng to [], and consderng that some of tracng modes would not be used durng some perod, the global updates of observaton vector and covarance matrx s P( t t) = ( P ( t t ) + 3 = xˆ( t t) = P( t t) *{ P 3 = w [ P w [ P ( t t) xˆ ( t t) ( t t) P ( t t )]) ( t t ) * xˆ( t t ) + P ( t t ) * xˆ( t t )]} where, w = 0, s the power of every tracng mode, whch s used as swtches of these tracng modes. The correlaton coeffcent calculated n regon-based tracng s used to determne the w, =,, 3 accordng to the followng crteron. () If the correlaton coeffcent s hgher than a threshold T, the occluson may not occur or occluson s very lght. As a result, the tracng results of modes based on actve contour and regon are sent to fuson center. But the tracng result of dstngushed ponts method s not used, for save the tme and accuracy. () If the correlaton coeffcent s smaller than T, the occluson may be occurrng. As a result, the tracng result of dstngushed ponts method s sent to fuson center. But others wll be dscarded. Global update Fuson center W W3 W Local update Occluson local update local update 3 occur? Global estmate Correlaton Actve contour based tracng mode coeffcent Regon based tracng mode Dstngushed ponts based tracng mode Image sequence obtaned by a statc camera Fgure : the structure of mult-mode tracng system 835
4 experence and result analyss In ths paper, two experences are presented. In the frst sequence, there s just one car n the scene. Usng the fuson system descrbed n fgure, the tracng result s presented, where we can see that w 3 s always zero, for there s no occluson. The tracng trajectory s shown n fgure, whch s very stable. Because of the complexty n the bacground, and the nose n the mage, the actve contour behaves not very well. But t can catch the man part of the target. (a) (b) (a) (c) (d) 0 (b) 0 8 8 9 9 0 0 Fgure (a)-(d) are tracng result usng actve contour method, whch denoted by whte curve. And usng regon based method, whch denoted by blac rectangle. The whte star on the car s the predcted poston of center by the fuson center. (f) s the tracng trajectory. The second sequence s about a hgh way, n whch a truc on the upper rght corner s traced. Snce there s car exceedng t, the occluson occurs. For a robust tracng, the dstngushed ponts based tracng s used, whch present a better result than regon based tracng. In fgure 3, the + are sgned for the center of target, f the tracng mode based on regon and actve contour are used. The star s sgned f the dstngushed ponted based method s appled. (e) (c) (c (d) (c) (d) 836
0 5 0 5 30 (e) 35 40 45 50 55 60 65 Fgure 3: The tracng of upper rght truc, whch s runnng left. (a) 7 th mage. A car begns to exceed the truc (b) th mage. the car occludes part of the truc. (c) 9 th mage, the car s under the truc, but dd not occlude the texture of truc. (d) 48 th mage. the car has exceeded the truc, and truc run stably. (e) the trajectory of truc from rght to left. When t s n occluson, the damond sgn s used; otherwse the crcle s used. The dmenson unt s 0 pxels. In Experence, from mage ~, there are occluson and bacground nose dsturbance. So the tracng mode based on dstngushed ponts s used. Durng ths perod, the modes based on regon and actve contour are runnng yet, but the results have not been sent to fuson center. The experence result shows that, even wth occluson, the mult-mode tracng system wll behave well by shftng the tracng mode. 5 Concluson In ths paper, a mult-mode tracng system based on dstrbuted fuson s bult, n whch three tracng modes are complementary to each other. Among these, regon based mode plays a ey role, whch affects other two tracng modes. Experence results show that ths tracng system tracs stable. It s the wor n future to mae the system sutable for more complex stuatons. [3] Zheng,Q.;Chellappa,R. Automatc feature pont extracton and tracng n mage sequences for unnown camera moton. Computer Vson, 993. Proceedngs., Fourth Internatonal Conference on, Aprl 993 Page(s):335 339 [4] Yu Zhong; Jan, A.K.; Dubusson-Jolly, M.-P. Object tracng usng deformable templates. Pattern Analyss and Machne Intellgence, IEEE Transactons on, Volume: Issue: 5, May 000 Page(s): 544-549 [5] M.Kass, A.Wtn, and D.Terzopoulos,Snaes :actve contour models. Int. Journal of Computer Vson, vol., pp.3 33,987 [6] Mao Wang, Joyce Evans, Laurence Hassebroo, Charles napp. A Multstage, Optmal Actve Contour Model. IEEE Trans. On Image processng. Vol 5,No.,996. pp:586-59. [7] Peterfreund, N. Robust tracng of poston and velocty wth Kalman snaes. Pattern Analyss and Machne Intellgence, IEEE Transactons on, Volume: Issue: 6, June 999 Page(s): 564 569 [8] Beymer, D.; McLauchlan, P.; Cofman, B.; Mal, J. A real-tme computer vson system for measurng traffc parameters. Computer Vson and Pattern Recognton, 997. Proceedngs., 997 IEEE Computer. Socety Conference on, 997. Page(s): 495-50 [9] Benjamn John Robbns.The Detecton Of D Image Features Usng Local Energy. PHD. Thess. 996 [0] W. T. Freeman and E. H. Adelson, The Desgn and use of steerable flters, IEEE Trans, Pattern Anal.Machne Intell., vol.pami- 3, pp. 89-906,99. [] Y.Bar-Shalo Multtarget Multsensor Tracng : Advanced Applcatons. Artech,990 [] A.Gelb,Appled Optmal Estmaton, the MIT Press,MA,and London,UK,974 Reference [] Gan Luca Forest, Vttoro Murno, Carlo Regazzon, Vehcle Recognton and Tracng from Road Image Sequences. IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 48, NO., JANUARY 999 [] Bascle, B.; Derche, R. Regon tracng through mage sequences. Computer Vson, 995.Proceedngs,Ffth Internatonal Conference on, 995 Page(s): 30 307 837