Distributed Multi-Target Tracking In A Self-Configuring Camera Network

Transcription

1 Dstrbuted Mult-Target Trackng In A Self-Confgurng Camera Network Crstan Soto, B Song, Amt K. Roy-Chowdhury Department of Electrcal Engneerng Unversty of Calforna, Rversde {cwlder,bsong,amtrc}@ee.ucr.edu Abstract Ths paper deals wth the problem of trackng multple targets n a dstrbuted network of self-confgurng pan-tltzoom cameras. We focus on applcatons where events unfold over a large geographc area and need to be analyzed by multple overlappng and non-overlappng actve cameras wthout a central unt accumulatng and analyzng all the data. The overall goal s to keep track of all targets n the regon of deployment of the cameras, whle selectvely focusng at a hgh resoluton on some partcular target features. To acqure all the targets at the desred resolutons whle keepng the entre scene n vew, we use cooperatve network control deas based on mult-player learnng n games. For trackng the targets as they move through the area covered by the cameras, we propose a specal applcaton of the dstrbuted estmaton algorthm known as Kalman-Consensus flter through whch each camera comes to a consensus wth ts neghborng cameras about the actual state of the target. Ths leads to a camera network topology that changes wth tme. Combnng these deas wth sngle-vew analyss, we have a completely dstrbuted approach for mult-target trackng and camera network self-confguraton. We show performance analyss results wth real-lfe experments on a network of 10 cameras. 1. Introducton Networks of vdeo cameras are beng nstalled n many applcatons, e.g., vdeo survellance, natonal and homeland securty, asssted lvng facltes, etc. It s natural to expect that these camera networks would be used to track targets at multple resolutons, e.g., multple people, a sngle person, a face. For effcency and maxmum resource utlzaton, t s desrable to actvely control the cameras so as to track the targets based on the requrements of the scene The authors were supported by NSF grants ECS and CNS , and ARO grant W911NF The authors would lke to thank Anesdo Fong for help wth the expermental evaluatons. beng analyzed. It would be prohbtvely expensve to have a statc setup that would cater to all possble stuatons. Currently, smlar applcatons try to cover the entre area or the most mportant parts of t wth a set of passve cameras but have dffculty n acqurng hgh resoluton shots selectvely. It s also desrable that the trackng and control mechansm be dstrbuted due to constrants of bandwdth, secure transmsson facltes, and dffculty n analyzng a huge amount of data centrally. In such stuatons, the cameras would have to act as autonomous agents and decsons would have to be taken n a dstrbuted manner. However, to be able to track all the targets n an area under survellance accurately, the cameras should be workng cooperatvely wth each other. Ths s because each camera s parameter settngs ental certan constrants on other cameras. Also, f a target s observed by multple cameras, there should be a consensus on the state (e.g., poston) of the target even f each camera s an autonomous agent Problem Descrpton The overall goal of ths paper s to develop a dstrbuted mult-target trackng and camera network control framework to observe and keep track of all targets at the desred resolutons n an actve camera network. We consder a network of synchronzed calbrated cameras wth pan, tlt, and zoom capabltes. Each camera has an embedded processng unt for local processng of the sensed vdeo data. Snce the cameras are calbrated, they can determne through ther homographes the poston of a target on a ground plane. Some of the cameras n the network may have overlappng felds of vew and a target may therefore be vewed by several cameras smultaneously. Due to naccuraces n calbraton and sngle-vew target trackng methods, the dfferent cameras vewng the same target wll not have exactly the same measurement of the target s state on the ground plane. Consequently, t s necessary for the cameras to collaborate to reach a consensus on the actual state of the target. In our framework, ths consensus must be reached only through pont-to-pont communcatons between neghborng cameras wthout the use of any central processng unt /09/$ IEEE

2 Fgure 1. A dagrammatc representaton of the proposed dstrbuted trackng and control system wth three neghborng smart cameras. As the targets move from one camera s FOV nto another, and as the cameras change ther parameters, the dstrbuted trackng system must also be able to keep track of the targets n a seamless way. Also, a camera may have to change ts parameters n coordnaton wth other cameras so as to keep the targets maged at the gven desred resolutons. All ths requres the trackng algorthm to be robust to dynamcally changng camera network topologes Overvew of soluton strategy We propose a specal applcaton of the Kalman- Consensus Flter presented n [10] to solve the problem of fndng a consensus on the state (poston and velocty) of multple targets n a dynamc camera network wth possbly overlappng felds of vew. In our proposed dstrbuted trackng algorthm, the cameras wll only communcate wth ther neghbors and ther specfc communcaton lnks wll vary n tme as the targets move through the area under survellance. Detals of the trackng approach are provded n Secton 3. The cooperatve and dstrbuted nature of ths problem also leads us to explore a game theoretc soluton for the camera control problem as detaled n Secton 4. Ths s acheved by the optmzaton of local sensor utlty functons leadng to an optmal value for a global utlty,.e. keepng vew of all targets at an acceptable resoluton and some at hgh resoluton. Fgure 1 shows an overvew of our dstrbuted multresoluton trackng system n a network of self-confgurng cameras. Each of the three neghborng smart cameras n ths fgure has ts own embedded trackng module, control module and Kalman-Consensus flter. The trackng module receves the vdeo data from the camera and performs the trackng of the targets n ts FOV. Snce the camera s calbrated, the trackng module can determne the (nosy) poston of each target on the ground plane. The Kalman- Consensus flter then uses ths target poston nformaton together wth poston and velocty nformaton from other neghborng cameras to come to a consensus wth these cameras about the actual state of the target. The control module on the other hand, changes the parameters of the camera as necessary to track the dfferent targets at the desred resolutons whle also keepng the entre area n vew. The parameter change s decded based on the parameters of neghborng cameras followng the gametheoretc framework presented n Secton 4. The parameters of the neghborng cameras are obtaned through a negotaton process. The Kalman-Consensus flter and control module run ndependently and asynchronously n each smart camera. As can be seen n Fgure 1, each camera exchanges only nformaton regardng ts own PTZ parameters and states of the targets that are n ts FOV. There s no vdeo nformaton beng sent to any other camera or to a central processng staton. It should also be ponted out that although the control module communcates wth all of ts neghbors, the Kalman-Consensus flter n our framework communcates only wth a subset of the neghborng cameras. Ths s advantageous because the state nformaton exchange for the consensus trackng requres a hgher data rate than the control negotaton nformaton. By reducng the number of neghborng cameras that need to communcate wth each other durng the consensus trackng, we reduce the overall communcaton bandwdth requred for the system. As can be seen clearly, ths entre framework leads to a completely dstrbuted approach for mult-target trackng and camera self-confguraton. 1487

3 1.3. Relaton to prevous work Some recent work has dealt wth networks of vson sensors, namely computng the statstcal dependence between cameras, computng the camera network topology, trackng over the network, and camera handoff [3, 5, 6, 12, 14]. There has also been recent work on trackng people n a mult-camera setup wth overlappng felds of vew [2, 4]. However, these methods are not totally dstrbuted and do not deal wth the the problem of trackng and control n an actve camera network. In [7], a dstrbuted target trackng approach usng a cluster-based Kalman flter was proposed. Here, a camera s selected as a cluster head whch aggregates all the measurements of a target to estmate ts poston usng a Kalman flter and sends that estmate to a central base staton. Our proposed trackng system dffers from ths method n that each camera n a neghborhood has a consensus-based estmate of the target s state and thus there s no need for addtonal computaton and communcaton to select a cluster head. Furthermore, we consder a dynamc camera network n whch, as the cameras change ther parameters, the targets are beng kept track of seamlessly. As wll be descrbed n Secton 3, we apply n a specal way the dstrbuted Kalman- Consensus flter [10] whch has been shown to be more effectve than other dstrbuted Kalman flter schemes. Consensus schemes have been ganng popularty n computer vson applcatons nvolvng multple cameras [15]. A related work that deals wth trackng targets n a camera network wth PTZ cameras s [11]. Here, the authors proposed a mxture between a dstrbuted and a centralzed scheme usng both statc and PTZ cameras n a vrtual camera network envronment. Our approach to camera control and trackng, however, s completely dstrbuted usng consensus algorthms and a game-theoretc framework [1]. Prelmnary work on ths game theoretc approach for camera control was presented n [13]. However, our goal n that paper was merely to cover an entre area, and there was no attempt to address the trackng problem, whch s crtcal for a dstrbuted approach to work. In ths paper, we also show real-lfe experments n a network of 10 cameras. The remander of ths paper s organzed as follows: Secton 3 presents our dstrbuted target trackng approach through conensus. Secton 4 states the problem of dstrbuted camera control and ts soluton n game theoretc terms. Our expermental results are presented n Secton 5. We summarze our work n Secton A Revew of Dstrbuted Estmaton and Cooperatve Control n A Sensor Network We brefly revew some basc concepts related to dstrbuted consensus and cooperatve control n a sensor network that are drectly relevant to our work Dstrbuted state estmaton In the mult-agent systems lterature, consensus means to reach an agreement regardng a certan quantty of nterest that depends on the state of all sensors n a network. There s no central unt that has access to all the data from the sensors. Consequently, a consensus algorthm s an nteracton rule that specfes nformaton exchange between a sensor and ts neghbors so that all the nodes reach a consensus. The nteracton topology of a network of sensors s represented usng a graph G = (V, E) wth the set of nodes V = {1, 2,..., n} and edges E V V. Each sensor node = 1,..., n s assocated wth a state x R. Reachng a consensus means asymptotcally convergng to a onedmensonal agreement space characterzed by the equaton x 1 = x 2 =... = x N. There have been recent attempts to acheve dynamc state estmaton n a consensus-lke manner. In contrast to a central Kalman flter where state nformaton comng from several sensors s fused n a central staton, Dstrbuted Kalman Flters (DKF) compute a consensus-based estmate on the state of nterest wth only pont-to-pont communcaton between the sensors[10]. A dstrbuted Kalman flterng (DKF) strategy that obtans consensus on state estmates was presented n [10].The overall performance of ths socalled Kalman-Consensus flter has been shown to be superor to other dstrbuted approaches. It s on ths DKF strategy that we base our dstrbuted trackng algorthm. The mathematcal detals are presented n Secton 3.2. A thorough revew of consensus n networked mult-agent systems can be found n [9] Dstrbuted control through cooperaton The theme of cooperatve control has receved sgnfcant attenton n recent years, especally the desgn of autonomous vehcles wth ntellgent and coordnated acton capabltes to acheve an overall objectve. In [1], an autonomous vehcle-target assgnment problem n whch a group of vehcles are expected to assgn themselves to a set of targets to optmze a global utlty functon s consdered. However, rather than optmzng the global utlty functon drectly, the emphass was on desgnng vehcles that are ndvdually capable of makng coordnated decsons to optmze ther own local utltes, whch then ndrectly translated nto the optmzaton of a global utlty functon. Ths approach enabled the vehcles to operate n envronments wth lmted nformaton, communcaton, and computaton, and stll be able to optmze a global utlty autonomously. Ths problem was formulated as a mult-player game where each vehcle was nterested n optmzng ts own 1488

4 utlty. The noton of pure Nash equlbrum can be used to represent the target assgnments that are agreeable to the ratonal vehcles,.e. the assgnments at whch there s no ncentve for any vehcle to unlaterally devate. As has been shown n [8], the combnaton of usng the Wonderful Lfe Utlty (WLU) as the vehcle utlty and Spatal Adaptve Play (SAP) as the negotaton mechansm leads to an optmal assgnment of targets wthn the set of pure Nash equlbra. The precse defntons of the utlty functons and negotaton mechansms for our applcaton wll be provded n Secton Dstrbuted Mult-Target Trackng Through Consensus 3.1. Dynamc Network Topology As mentoned earler, we propose a specal applcaton of the Kalman-Consensus Flter presented n [10] to solve the problem of fndng a consensus on the state of multple targets n a dynamc camera network wth possbly overlappng felds of vew. However, unlke the sensor nodes n [10], the nodes n our camera network are drectonal sensors and a new method for establshng the network topology between the camera nodes must therefore be developed. Furthermore, snce the camera network s composed of cameras that change ther parameters and therefore ther felds of vew as needed, the network topology wll be dynamcally changng also. Let C be the set of all cameras n the network. We can then defne the subset of all cameras vewng target T l as Cl v C and the rest of the cameras as C p l C. Each camera C wll also have ts set of neghborng cameras C n C. Snce all the cameras can change ther PTZ parameters and have therefore several possble felds of vew, we defne the set C n as all the cameras wth whch C can potentally have an overlappng feld of vew. By defnton, t becomes clear then that for each C Cl v, t s true that Cv l {C n C },.e. all of the cameras vewng a specfc target are also neghbors. Note that the set of neghbors need not be geographcal neghbors. In our proposed dstrbuted trackng system, only cameras that have an observaton of a target T l wll communcate wth ther neghbors. Ths feature lmts the amount of data exchanged for the consensus trackng. By reducng the number of neghborng cameras that need to communcate wth each other durng the consensus trackng, we also reduce the power requrements of each camera node and the overall communcaton bandwdth requred for the system. Ths way of defnng the network connectons between the camera nodes brngs about a dynamc graph G l (k) = (V l (k), E l (k)) representng the network topology of the system wth respect to target T l at tme nstant k. When Algorthm 1 Dstrbuted Kalman-Consensus trackng algorthm performed by every C at dscrete tme step k. The state of T l s represented by x l wth error covarance matrx P l (see Sec. 3.2). Input: x l and Pl from tme step k 1 for each T l that s beng vewed by {C n C } do f C Cl v (.e. C s vewng T l ) then Obtan ground plane measurement z l wth covarance Rl Compute nformaton vector and matrx u l = Hl T R l 1 z l U l = Hl T R l 1 H l Send message m l = (ul, Ul, xl ) to neghborng cameras Cn end f Receve messages m j = (u l j, Ul j, xl j ) from all cameras C j Cl v Fuse nformaton matrces X and vectors X y l = u l j, Sl = U l j j C v l j C v l Compute the Kalman-Consensus state estmate M l = ((Pl ) 1 + S l ) 1 X ˆx l = xl + Ml (yl Sl xl ) + γml ( x l j xl ) j C v l γ = 1/( M l + 1), X = (tr(xt X)) 1 2 Update the state and error covarance matrx for tme step k end for P l Al M l AlT + B l Q l B lt x l Alˆx l there are multple targets n the area under survellance, there wll be a dstnct network topology for each target. Obvously, as T l moves through the area under survellance, G l (k) wll also change snce other cameras wll be vewng T l and only these cameras wll communcate wth ther neghbors thus creatng a dynamc network topology. The network topologes of all the targets wll also change when one or more cameras change ther parameters. As wll be explaned n the followng secton, ths defnton of network topology for dstrbuted trackng does not affect n any way the performance of the Kalman-Consensus flter presented next Kalman-Consensus trackng To model the moton of a target T l on the ground plane as observed by camera C, we consder a lnear dynamcal system wth process and sensng models x l (k + 1) = A l (k)x l (k) + B l (k)w l (k); x l (0) (1) z l (k) = H l (k)x l (k) + v l (k) (2) where w l (k) and v l (k) are zero mean whte Gaussan nose (w l (k) N (0, Q l ), v (k) N (0, R l )) and x l (0) s the ntal state of the target. We defne the state of the target at 1489

5 tme step k as x l (k) = (x l (k), y l (k), ẋ l (k), ẏ l (k)) T where (x l (k), y l (k)) and (ẋ l (k), ẏ l (k)) are the poston and velocty of target T l n the x and y drectons respectvely. x l s the state of T l based on the observatons n C only. The nosy measurement z l (k) at camera C s the sensed target poston (x l (k), yl (k)) on the ground plane based on the pre-computed homography. Our specal mplementaton of the Kalman-Consensus dstrbuted trackng algorthm s presented n Algorthm 1. Ths algorthm s performed dstrbutedly n each camera node C. At each tme step k and for each target T l, we assume to be gven the estmated target state x l and the error covarance matrx P l from tme step (k 1). Each C also knows ts set of neghbors C n from the nformaton used by the camera s control module. At tme step k = 0, the Kalman-Consensus flter s ntalzed wth P l = P 0 and x l = xl (0) = average of zl (k) s of neghbors vewng T l. The followng gves a verbal descrpton of Algorthm 1 performed at each C dstrbutedly for each T l that s vewed by {C n C }. If C s vewng a target T l, t determnes T l s ground plane poston z l and computes the correspondng nformaton vector and matrx wth the gven measurement covarance matrx R l and output matrx H l. C then sends a message m l to ts neghbors whch ncludes the computed nformaton vector and matrx, and ts estmated target state x l at prevous tme step (k 1). C then receves smlar messages m j only from the cameras n ts neghborhood that are also vewng T l. The nformaton matrces and vectors receved from these messages, and ts own nformaton matrx and vector f C s vewng T l, are then fused and the Kalman-Consensus state estmate s computed n a way smlar to the method proposed n [10]. Fnally, the ground plane state x l and error covarance matrx Pl are updated accordng to the assumed lnear dynamcal system Handoff, Network Reconfguraton and Fault Tolerance Through ths algorthm, each C has a consensus-based ground plane state estmate of each target that s beng vewed by ts neghborng cameras, even f C has never seen some of the targets. Snce we are assumng that the network of cameras as a whole s always coverng the entre area under survellance through the game theoretc control framework presented n Secton 4, the target wll always be seen by at least one camera. Also, by our defnton of neghborng cameras, a target T l wll always move from one camera C s FOV to the FOV of a neghborng camera C j C n. Therefore, C j can take over the trackng of T l and fnd the target correspondence n a seamless way snce t had knowledge of T l s ground plane poston through the consensus-trackng before t even entered ts FOV. Addtonal target features could be used to fnd the target correspondences n a cluttered scene. Furthermore, even as a camera changes ts parameters, t can also take over the trackng of the targets n ts new FOV mmedately snce t also knew the poston of the targets n ts neghborhood beforehand through the consensus-trackng algorthm. Another advantage of the fact that cameras have knowledge of all the targets n ther neghborhood s that n the event of a sudden falure of camera node C, the targets that were vewed by C are not suddenly lost by the camera network. The neghborng cameras can adjust ther parameters to cover the area that was left uncovered by C s falure and contnue wth the consensus-trackng algorthm wthout any nterrupton. We have also consdered the fact that a camera may take a short amount of tme to change ts parameters to a new poston. If no camera s vewng the target for the short amount of tme t takes for the cameras to come to a new set of parameters to cover the entre area, the target state estmate just follows the assumed lnear state equatons. Ths does not translate to a sgnfcant decrease n trackng performance as seen n our experments. In the next secton, we explan how the camera parameters are changed n order to keep the dfferent targets maged at the specfed resoluton. 4. Dstrbuted Self-Confguraton of Camera Network Usng Game Theory Our goal s to develop a dstrbuted strategy for coordnated self-confguraton of the camera network that reles on local decson-makng at the camera nodes, whle beng algned wth the sutable global crteron of observng multple targets at multple resolutons. For ths purpose, we use game theoretc deas that rely on mult-player learnng and negotaton mechansms Game Theoretc Motvaton Let us consder N t targets n the entre area of deployment and N c sensors that need to be assgned to these targets. The targets {T } are the opponents. A target loses and s elmnated from the game f t s captured n an mage at the desred resoluton. Our team of cameras scores each tme t obtans a desred resoluton mage of a target. A camera C wll select ts own assgnment profle, a, (.e. the set of targets to track) by optmzng ts own utlty functon U C (a ). Our problem s to desgn these utlty functons and approprate negotaton procedures that lead to a mutually agreeable assgnment of targets resultng n meetng the global crteron Choce of utlty functons Target Utlty: Let V l be the value of observng target T l and p l the probablty that target T l s acqured at the 1490

6 desred resoluton by camera C engagng T l. Then, we defne the utlty of observng T l usng a partcular assgnment profle A = a to be [ U Tl (A) = V l 1 ] (1 p l ). (3) p l s defned as p l = { 1 e λx l f x l > r 0 /r max, (4) 0 otherwse where r 0 s the mnmum acceptable resoluton n terms of target pxel heght at whch the targets should be vewed at and r max s the heght n pxels of C s mage plane. If r l s the resoluton at whch T l s beng vewed at by C, then x l = r l /r max. The term λ can be changed accordng to how well the sngle-vew trackng algorthm performs as the heght of the target on the mage plane ncreases or decreases. Global Utlty: From the target utlty functon, we can now defne the global utlty functon as the sum of the utltes generated by observng all the targets,.e., U g (A) = T l U Tl (A). (5) Camera Utlty: In our applcaton, we use what s known as Wonderful Lfe Utlty (WLU). In WLU, the utlty of a sensor observng a partcular target s the margnal contrbuton to the global utlty as a result of ths acton,.e., the sensor utlty s the change n the global utlty as a result of that sensor observng that partcular target as opposed to not observng t. Snce each camera C n our dstrbuted camera network can nfluence only the target assgnments of ts neghbors C n, we wll defne the camera utlty dependng only on the assgnments of ts neghborng cameras,.e. U C (A) = U g (A) U g (a ) = U Tl (A) U Tl (a ), (6) T l C n where a = A a s the assgnment profle of all the cameras except C. As shown n [8], the WLU utlty leads to a potental game wth the global utlty functon as the potental functon, and hence they are algned wth the global utlty. Ths ensures that the resultng set of targets that are chosen wll be ncluded wthn the set of pure Nash equlbra Negotaton mechansms Persstently observng the objects of nterest n a dynamc settng requres negotaton mechansms between the dfferent sensors, allowng them to come up wth the strategc decsons descrbed above. Each sensor negotates wth other sensors to accurately predct ther team-mates parameters, and decde ts own acton. A partcularly appealng strategy for ths problem s Spatal Adaptve Play (SAP) [8]. Ths s because t can be mplemented wth a low computatonal burden on each camera and leads to an optmal assgnment of targets wth arbtrarly hgh probabltes for the WLU descrbed above. In a partcular step of the SAP negotaton strategy, a camera C s randomly chosen from the pool of cameras n the network accordng to a unform dstrbuton and only ths camera s gven the chance to update ts proposed parameter settngs. Let us now consder the case where a specfc applcaton requres the trackng of a target Tl h at a hgh resoluton r h. When a camera C s chosen to update ts parameters at any negotaton step, t wll frst determne f t can adjust ts parameters to vew the target at the desred hgh resoluton. If t can not do so, C wll smply contnue wth the negotaton cycle maxmzng ts own utlty and sendng ts new parameters to ts neghbors C n as presented above. If C s however able to vew Tl h, t wll set ts parameters accordngly, take over the task of vewng the target and transmt ts parameters to ts neghbors C n. When C s not able to change ts parameters to vew Tl h, t wll send a handoff flag to ts neghbors C n ndcatng that another camera needs to take over the task of vewng the target. C then returns to the negotaton mechansm maxmzng ts own utlty U C. It s to note that the tme between negotaton steps does not need to be the same as the dscrete tme step sze n the Kalman-Consensus trackng descrbed n Secton 3 snce the two processes run parallelly and asynchronously to each other n each smart-camera. 5. Expermental Results We tested our approach for trackng and camera selfconfguraton n a real camera network composed of 10 PTZ cameras lookng over an outdoor area of approxmately sq. feet. We dvded the area nto contguous blocks and each of the centers of those blocks was consdered a vrtual target of heght 1.70m. Therefore, f the camera network as a whole s coverng all of the vrtual targets n each block at an acceptable resoluton r 0 = 40 pxels n the vertcal drecton, all of the actual targets n the area under survellance are also beng vewed at an acceptable resoluton. In the area under survellance, there were 8 targets n total that were to be tracked usng our dstrbuted Kalman- Consensus flterng approach. Fgure 2 shows the trackng and control results as vewed by each camera at 4 tme nstants. Due to space constrants, we show only 4 (C 1,..., C 4 ) of the 10 cameras n the camera network. 1491

7 (a) k = 64 (b) k = 74 (c) k = 93 (d) k = 151 Fgure 2. Each column shows one of 4 of the 10 cameras at four tme nstants denoted by k. A target marked wth a box s always tracked at a hgh resoluton. Note that the camera parameters are changng to acheve ths whle coverng the entre area at an acceptable resoluton. The other targets are tracked usng the Kalman-Consensus flterng approach, but are not marked for clarty. The vdeo s avalable as supplementary materal and at amtrc/cameranetworks.htm. At an ntal state, all of the cameras have random PTZ parameters and are not coverng the entre area under survellance. After the cameras start runnng ther control modules, they converge to the fnal confguraton seen partly n Fgure 2(a) coverng the entre area. As the targets are observed n ths area, the sngle-vew trackng module n each camera determnes the ground plane poston of each target n ts FOV and sends that nformaton to the KalmanConsensus flter whch processes t together wth the nformaton receved from the Kalman-Consensus flters of neghborng cameras as descrbed n Secton 3. Fgure 2(b) shows the nstant when a camera C3 zoomed nto a target Tlh that was marked for beng tracked at a hgh resoluton. Snce C3 changed ts parameters and left some area t was coverng uncovered, the other cameras n the network adjust ther own parameters to cover that uncovered area followng the negotaton mechansms descrbed above. These changes n parameters can be seen n Fgure 2(b) and (c) for cameras C3 and C4. As shown n Fgure 2(d), when C3 s not able to change ts parameters to keep trackng Tlh, C1 takes over the trackng and C3 changes ts parameters to cover some area that was left uncovered by C1. It s to note that every tme a target goes from one camera s FOV nto another one, or when a camera changes ts parameters, the network topologes for the targets change also. 1 Fgure 3 shows the dstrbuted Kalman-Consensus trackng trajectores for the 8 targets. The observatons of the dfferent cameras are shown n a lght gray color. As can be seen, even though the homography-based observatons are nosy and the network topology s changng constantly, the Kalman-Consensus flter n each camera comes to a smooth consensus about the actual state of the target. 1 The code for a smulaton of the camera network control and Kalman-Consensus flter s avalable at amtrc/cameranetworks.htm. 1492

8 topology for trackng persstently multple targets across several camera vews n an area under survellance vewed by a dynamc camera network. A camera control framework based on game theoretcal deas allowed for vewng some targets at a hgh resoluton whle keepng the entre area under survellance covered at an acceptable resoluton. References Fgure 3. Dstrbuted Kalman-Consensus trackng trajectores for 8 targets. Observatons from all cameras are shown n a lght gray color. Fgure 4. Trackng results on the ground plane for one of the targets. Fgure 4 shows the dstrbuted trackng results n the y ground plane drecton for one of the targets. The dots correspond to the observatons from the dfferent cameras vewng the target whle the sold lne s the consensusbased estmate. As can be expected, the observatons are dfferent for each camera due to calbraton and sngle-vew trackng naccuraces. The vertcal dashed lnes ndcate the tme nstants of change n the dynamc network topology wth respect to the target,.e. when the target goes nto or out of a camera s FOV. As can be seen clearly, even though dfferent combnatons of cameras vew the target at dfferent tme nstants, the Kalman-Consensus flter fnds an estmate of the target s poston seamlessly at all tmes. 6. Concluson We presented n ths paper a robust approach to dstrbuted mult-target trackng n a network of selfconfgurng cameras. A dstrbuted Kalman-Consensus flterng approach was used together wth a dynamc network [1] G. Arslan, J. Marden, and J. Shamma. Autonomous Vehcle-Target Assgnment: A Game-Theoretcal Formulaton. ASME Journal of Dynamc Systems, Measurement and Control, 129(5), September [2] W. Du and J. Pater. Mult-camera People Trackng by Collaboratve Partcle Flters and Prncpal Axs-Based Integraton. In Asan Conf. on Computer Vson, [3] S. Khan, O. Javed, Z. Rasheed, and M. Shah. Camera Handoff: Trackng n Multple Uncalbrated Statonary Cameras. In IEEE Workshop on Human Moton, [4] S. Khan and M. Shah. A Multvew Approach to Trackng People n Crowded Scenes Usng a Planar Homography Constrant. In Euro. Conference on Computer Vson, [5] Y. L and B. Bhanu. Utlty-based dynamc camera assgnment and hand-off n a vdeo network. IEEE/ACM Intl. Conf. on Dstrbuted Smart Cameras, [6] D. Marks, T. Ells, and J. Black. Brdgng the Gap Between Cameras. In IEEE Conf. on Computer Vson and Pattern Recognton, [7] H. Mederos, J. Park, and A. Kak. Dstrbuted object trackng usng a cluster-based kalman flter n wreless camera networks. IEEE Journal of Selected Topcs n Sgnal Processng, 2(4): , Aug [8] D. Monderer and L. Shapley. Potental games. In Games and Economc Behavor, [9] R. Olfat-Saber, J. Fax, and R. Murray. Consensus and Cooperaton n Networked Mult-Agent Systems. Proceedngs of the IEEE, 95(1): , Jan [10] R. Olfat-Saber and N. F. Sandell. Dstrbuted trackng n sensor networks wth lmted sensng range. Proceedngs of the Amercan Control Conference, June [11] F. Quresh and D. Terzopoulos. Survellance n Vrtual Realty: System Desgn and Mult-Camera Control. IEEE Conf. on Computer Vson and Pattern Recognton, [12] B. Song and A. Roy-Chowdhury. Stochastc Adaptve Trackng n a Camera Network. In IEEE Intl. Conf. on Computer Vson, [13] B. Song, C. Soto, A. K. Roy-Chowdhury, and J. A. Farrell. Decentralzed camera network control usng game theory. Workshop on Smart Camera and Vsual Sensor Networks at ICDSC, [14] K. Teu, G. Dalley, and W. Grmson. Inference of Non- Overlappng Camera Network Topology by Measurng Statstcal Dependence. In IEEE Intl. Conf. on Computer Vson, [15] R. Tron, R. Vdal, and A. Terzs. Dstrbuted pose averagng n camera networks va consensus on SE(3). IEEE/ACM Intl. Conf. on Dstrbuted Smart Cameras, Sept