- PDF Free Download

Transcription

1 Motion detection, labeling, data aociation and tracking, in real-time on RISC computer Lionel Lacaagne ;, Maurice Milgram, Patrick Garda Laboratoire de Intrument et Sytµeme Univerit e Pierre et Marie Curie BC place Juieu, 7 Pari - FRANCE lionel j garda@li.juieu.fr maum@ccr.juieu.fr IMASYS Le quadral bi rue E. Nieuport 9 Surene - FRANCE lionel@imay.fr Abtract - Thi article introduce ome algorithm for motion detection and tracking. To be able to track moving ob ject, the analyi rate mut be fat enough in order to keep the conidered movement low (and the feature of the ob ject not changing too fat). We are preenting a Markov Random Field baed motion detection algorithm running in real time on Pentium II and DSP C6x. For the medium level, we have developed a fat labeling algorithm, a benchmark i provided. The high level i fat becaue the previou proceing tep are running at a fater rate than the video rate. I. Motion Detection The motion detection (MD) i the bae of all ytem for tracking ob ject and image compreion. The MD algorithm are mainly belonging to two clae: optical ow and correlation. Thee method have been widely explored, everyone of them ha ome advantage and drawback. The implet method i an image di erence, where a igni cant variation of the grey level i aociated to a movement. At the ame time, Markov Random Field baed algorithm have aert themelve in a lot of image proceing area for regulariing ill poed problem( edge detection, region egmentation, retoration)[6]. If their main advantage i the robutne, their main drawback i their CPU conumming due to the huge amount of calculation. Thi lead reearcher to tudy a lot of olution to peed their execution up, a Parallel Machine, or dedicated architecture [][][]. We decribe the ue of MRF for enhancing an image di erence. After explaining the di erent kind of optimization, we how how to reach real time on RISC and DSP. A. A MRF approach The aim of the markovian proce i to improve the quality of the image di erence with a relaxation algorithm. Becaue of change of illumination, thi image i very noiy: a lot of pixel are labelled a in motion and the region are not lled. The energy model ha been introduced by the Li-Grenoble lab [][]. Let I t, be the grey level image at the moment t, and O t the obervation (O t = ji t I t j). Then threholding O t initialize the etimated eld of label Ê t : The background/movement ite are labeled =. The Iterated Conditional Mode (ICM g) i then applied to nd the relaxed label eld E t from Ê t, Ê t and E t. Relaxation being determinitic, the algorithm converge in few iteration ( xed to four), to the rt local minimum, o the initialization i very important. The traight forward image difference may be enhanced by a maximum likelihood [7], with or without con dence (CPU expenive).

2 Buffer I t- I t- I t O t - O t Binariation Binariation E t- Ê t- Ê t E t- ICM Figure { Relaxation algorithm V p e t ; e t V f e t ; e t+ p if e t = = e t + p if e t 6= et = f if e t = et+ + f if e t 6= et+ With the Iing model (pin up/pin down : +/- ): P u m = cc xt xt r pxt xt f x t xt+ with x t, the pin of the ite x, at t. In order to make the trace of I t to diappear, and to ll the overlapping, area due to the image di erence, it i convenient to repect the inequality: p < < f Experimental tet demontrate that the required parameter ( p ; ; f ; ) do not have to be tuned according to the image equence. They are et to the value: p = ; = ; f = ; = B. Energy function, clique. The energy model u ued i the um of two energie: u m model energy that i a regulation term which enure a patiotemporal homogeneity and which alo expree the relation with the neighboring ite, and u a adequation energy which prevent the model to drift too far from the initialization. u (o ; e ) = u m (e ) + u a (o ; e ) u m (e ) = P cc V c (e ;e r ) u a (o ; e ) = [o ¾ ª (e )] if background ª (e ) = > if mouvement homogeneou overlapping area I t- trace Figure { O t = ji t I t j I t C. Optimization t- t t+ Figure { patio-temporal clique Clique aociated to u m energy i a patiotemporal and econd order clique (Fig ). The patial potential function V i: V c (e ; e r ) = V (e ; e r )+V p (e t ; e t )+V f (e t ; e t+ ) V (e ; e r ) = if e = e r + if e 6= e r The temporal potential V p and V f, aociated to the pat and future clique are:. The optimization technique The mot common ued technique i the 'loopunrolling' (LU). It conit in the duplication of the loop body, but alo, when it i poible to make ome computation in 'parallel'. Even o, the C6x pipeline i not alway full. The oftware pipelining (SP) can olve thi problem by maximizing the ue of the di erent unit.. Algorithm characteritic Two conecutive mak overlap on two column. So the computation of the patial energy i then plitup in three column, and only one new column i computed intead of the three previouly required. The entire computation i performed by Look-Up

3 claical loop loop begin loop unrolling oftware pipelinig on a C6x (the indexe correpond to the delay with the current index t). loop end A B C = t + t = t + t a =A[j] c =C[j] b = B[j] t=a+c += t += b empty pace p = P[j-] up = (-p )beta p u = (8- )beta um =up +uf Figure { loop-unrolling, oftware pipelining f = F[j-] uf = (-f )beta f um += u Table (LUT) to avoid CPU intenive multiplication. The background/movement ite are labeled = rather than =. Intead of comparing the tate of each ite to the central ite e, we jut take into account ite which tate are (p, and f for the pat, preent and future image). So if the central ite tate i the patial energy i: u m = (8 ) + ( p ) p + ( f ) f, ele u m = u m If u m + u a < u m + u a, the tate i et to otherwie it i et to. The change i performed whatever the previou tate i. The tet can be impli ed by factoring the model and adequation term: u m < u a u a = ¾ (o ). Thi a priori minor modi cation can ave cycle on a total of 6 on DSP C6x, that i to ay 7%!. Implemented optimization The algorithm ha been implemented on a Pentium II MHz, and ha been etimated (from the number of cycle of the loop) for the TI-C6 at MHz. On the Pentium the number of available regiter i inu±cient to perform the Loop-Unrolling. So thi i the 8-accee-per-point verion, but with a LUT, that ha been implemented. On the C6x there are enough regiter. A Loop-Unrolling of order i performed. Becaue the model energy i computed fater with multiplication ( cyle) than with LUT ( cycle) we ue multiplication for u m and LUT for u a. The gure give the cheduling o = O[j-] delta_ua =DUa [o ] tet Figure { C6x cheduling B[j-] = b Per point, and on a C6, the etimated time, depending on the performed optimization (none, Loop-Unrolling, or Software Pipelining) are: optimization technique none LU SP intruction cycle ipc.7.6. % charge 6 MIPS A comparion of the di erent architecture implementing the ICM relaxation i given in the gure 6. Standard RISC proceor, programmed in C, achieve real time execution for image ize larger than 6 (up to ), and xed point DSP, in aembly language for image ize larger than (up to 7 ). II. Light Speed Labeling: a fat and new algorithm Thi paragraph introduce a fat algorithm for labelling binary image.it peed i almot independant from data. Moreover it tructure make the 8-connected verion a fat a the -connected ver-

4 Lab Architechture # chip f(mhz) image ize rate ² a cyclic implementation of equivalence intead of a \ at" implementation. Li Grenoble Li Grenoble Lamea Li Pari CNAPS 6 M 96 Tranviion (T9) C ~6 (x88) 6 8 Here are the di erent tep of the algorithm. The explanation will be illutrated by the example of the gure 8. Li Pari Li Pari Pentium II C6 (imulation) 6 7 j Figure 6 { di erent implementation L i- L i ion. The propoed algorithm i optimum with the number of created label, and it can be eaily and e±ciently parallelized. A. Claic labeling, RLC-baed labeling Figure 8 { Line L i, L i. Computation of ER i and RLC i B A C X B A X D ER i- j ER i Figure 7 { and 8-connected neighbor labeling Standard algorithm ue either the pixel approach (Fig 7: comparion of the current label, with the neighboring pixel) or the Run Length Coding (earch for an adjacent egment in order to nd out the connectivity). Both of them involve a great amount of tet, and, at the CPU level thi lead to many pipeline tall (waite of time) becaue the tet reult i, of coure, unpredictible. Latly and it i not it leat drawback, the pixel labeling produce many uele label. B. LSL algorithm The main idea in LSL i to replace all the tet (label comparion, fuion ort for egment interection) by a et of imple intruction without (or with much le) tet. Thee intruction are fat becaue pipelinable. The peed of LSL i baed on key point: ² intenive ue of RLC ² introduction of \relative" and \abolute" label for a direct acce (LUT) to the label of the interected egment Figure 9 { table ER The ER i table hold, for each line L i, the aociated relative label e r of each egment (object with an odd number) and each non-egment (background with an even number). RLC i- RLC i Figure { table RLC The RLC i table hold the \begin" and \end" indexe of each egment [a::b]. The j th egment of relative label e r = j + ha the boundarie a = RLC i [j], b = RLC i [j + ]. The current egment ha an interection with the egment numbered e r = ER i [a] e r = ER i [b]. The label e r / e r are enventually adjuted to point to the next / previou egment and not to the background: To perform a 8-connected labeling, which i equivalent to tet the diagonal label (NW, NE), it i enough to increment a by and to decrement b by, jut before looking up the ER i table. That make

5 the 8-connected verion a fat a the -connected verion.. Computation of ERA i L i- L i abolute label relative label Figure { abolute and relative label The ERA i aociate the relative and abolute label of a line. The table i calculated during the topdown label propagation. The relative label i aociated with the abolute label of the rt interected egment: e r = a = RLCi [] = ) a = b = RLC i [] = 9 ) b = 9 e a = ERA i (e r ) =, ERA i (e r ) = e a er = e r = 6 ) e r =. Computation of EQ To olve equivalence problem (computing the tranitive cloing of a graph i a complex problem) like adding an item to a cla, or more generally to fuion two clae, the equivalence clae have been implemented with cyclic graph in the EQ table. ++ Figure { equivalence cycle During the fuion, e a,mut be aociated with the other abolute label e ak of the interected label. e r k [e r + ::e r ], e ak = ERA i [e rk ] fuion(e a ; e ak ) with the example of the previou ection: e r k [e r + ::e r ] e r k f; g, e ak f; g for the abolute label, we have the equivalence. The equivalence of the label will be proceed like in gure.. Computation of EA i The abolute label of Li are obtained by LUT: EA i = ERA i (ER i ). Packing label and relabeling At the end, label are packed (to get continuou number) from EQ to A. The whole image i then updated: EA i = A(EA i ) C. Benchmark of LSL The algorithm ha been compare to claic pixel baed algorithm, eventually enhanced by cyclic equivalence. All verion ue 6-bit label. LSL i alo implemented in 8-bit becaue ometime the number of label i inferior to 6. We teted the algorithm on cae of labeling : mall/big region few/many character. We give the reult in m for -connected and 8-connected verion. The image ize i. bolt logo cap7 cap8 bolt logo C 8C C 8C pixel 6 6 pixel+cycle 68 LSL 6bit LSL 8bit LSL v pixel gain cap 7 cap 8 C 8C C 8C pixel pixel+cycle 9 LSL 6bit LSL 8bit - - LSL v pixel gain Our algorithm outperform by a factor from. to. pixel-labeling for the type of image ued in real motion detection. Even for OCR, LSL i a bit fater.

6 III. Region matching The matching ytem remain imple. It can not track neither two croing region, nor manage an object plitted into many region. But it i fat and it will be the bae to compare to clever ytem uing arti cial intelligence [8]. E E IV. Concluion I t For two conecutive image (I t ; I t ), the interection area i computed (in grey on the Fig I t ). for each region thee urface are orted into decending numerical order. Aociating label i then equivalent to earch the bet matching all over the tree. In thi tree a node correpond to a peci c matching between two region of two conecutive image, and a branch correpond to a et of matching. The choen ub-tree i thi which maximize the um of the interected urface. Thank to the ort, the bet ub-tree i located on the left of the tree, even it i the leftiet ub-tree of the tree. The following image are extracted from the famou \taxi" equence. Figure I and I are the original image, gure Ê ; Ê are the abolute difference O t binarized with a threhold value of, and gure E ; E are the ICM relaxed image. I t I I Ê Ê In thi article, we have hown how to get a real time and a robut motion detection baed on MRF on a RISC and DSP. We have propoed an new labeling algorithm, fater than the claical one. Then, becaue thee algorithm are fat and becaue we get good reult, it remain enough time to perform a region matching and a tracking. Becaue all of thee algoritm have been optimized, the whole proceing i running in real time, at the video rate for 6 6 image. Reference [] R. Azencott \Simulated annealing: parallelization technique" John Whiley 99 [] A. Bellon \D etection et uivi de v ehicule en mouvement, impl ementation parallµele ur un ytµeme exp erimental µa m emoire ditribu ee", thµee LAS- MEA, Clermont Ferrand, Oct 997. [] A. Caplier \Modµele markovien de d etection de mouvement dan le equence d'image: approche patio-temporelle, mie en oeuvre temp r eel" thµee INPG, Grenoble 99. [] A. Caplier, F. Luthon, C. Dumontier \Algorithme markovien de d etection de mouvement, mie en oeuvre temp r eel" GRETSI 99. [] J.P. Cocquerez, S. Philipp \Analye d'image: ltrage et egmentation", Maon 99. Temp de calcul ur Sparc et CM (INRIA-Sophia), p. [6] F. Heitz, P. Bouth emy \Multimodal etimation of dicontinuou optical ow uing MRF" IEEE, Tran. on PAMI-, N-, Dec 99. [7] Y.Z. Hu, H.H. Nagel, G. Reker \New likelihood tet method for change detection in image equence" CVGIP 6, 7-6, 98 [8] P.H. Winton \Arti cial Intelligence" Adion- Weley, 98 6