A Real-time Visual Tracking System in the Robot Soccer Domain

Transcription

1 Proceedgs of EUEL obotcs-, Salford, Eglad, th - th Aprl A eal-tme Vsual Trackg System the obot Soccer Doma Bo L, Edward Smth, Huosheg Hu, Lbor Spacek Departmet of Computer Scece, Uversty of Essex, Wvehoe Park, Colchester CO 3SQ, UK Abstract Ths paper presets a real-tme vsual trackg system the robot soccer doma. The detals of the Essex Wzards robot football team ad ts cotrol system are descrbed. A real-tme vsual trackg algorthm has bee developed. Some tal results of the mage pre-processg ad vsual trackg algorthms real evromet settg are gve to show the performace of the system.. Itroducto obot soccer provdes a good research platform the area of mult-aget systems, whch volves automatc cotrol, wreless commucato, mage processg ad artfcal tellgece. May researchers have studed ad got some valuable results from t [,,5,6,7,5]. It s as terestg as the real soccer game. Sce robots ru fast the ptch, the requremet for respose tme of the system cludg mage processg, team strategy ad commucato s very hgh. Ay delay of the system wll lead the team to lose the whole game. The touramet s oe of FIA games [7] ad ams to develop mult-aget systems to play the game of soccer ad complete varous team strateges agast each other [,]. The st MIOSOT World Champo was hold 996 ad the became a aual evet. The MIOSOT touramet has a smplfcato of the stadard rules of real soccer. It cossts of teams of three small robots followg commads from a cetralsed cotrol computer. Each robot has wreless lk to commucate wth the cetral computer to receve commads. They also have some I sesors used for object detecto. A overhead camera gets the cotuos mages of the ptch, whch are coverted to dgtal format ad processed by the software to dcate the postos of all the robots ad the ball []. Ths paper presets a real-tme vsual trackg system accordg to the MIOSOT soccer games requremet. I the ext secto a bref troducto of the robot soccer systems that were bult at Essex s preseted, cludg robot cofgurato, the team cotroller, vso system hardware ad software. I secto 3, mage preprocessg s brefly descrbed. The object detecto algorthm for our soccer robots s gve secto. The proposed object trackg ad predcto algorthms are detaled secto 5. Secto 6 presets some tal results for the realtme trackg of the ball ad robots. Fally, bref coclusos ad future work are summarsed secto 7.. The robot soccer system Accordg to the FIA rules, the team cossts of three small robots wth the sze of 7.5cm 7.5cm 7.5cm. Ths sze lmts the amout of the compoets oboard ad also restrcts the structure of the gearbox ad motor system. The cetral computer takes the jobs to perceve the evromet of the ptch, remote cotrol each robot ad co-ordate the team behavours. Wth the smple cotrol system o-board, each robot oly follows the commads from computer ad reacts to the local hazard. Fgure Essex Soccer obot The FIA rules defe blue ad yellow as the offcal team colour ad orage as the ball colour. Each team eeds to prepare other three colours to detfy each robot. The top of the robots must be marked wth ther team colour clearly. The ptch

2 colour s black ad t has some whte to dcate the border ad specal spots.. obot cofgurato The Essex soccer robot, as show fgure, has three ma modules: a motor-mechacs module, a cotrol module ad a sesor module. A 8MHz Motorola 68HCE s used as the CPU of the cotrol module ad t works the sgle chp mode wthout ay extedg AM or OM. The robots have two frot wheels dfferetally drve by two small DC motors wth gearbox. The motor speed s measured by the optcal ecoder. Four frared sesors are used to detect the obstacle or other robots to deal wth the emergg collsos that ca ot be reacted by the cetral computer. There s also a ball detector the frot pael of each robot. To adjust the compesato of the battery power, the robots have sesors to read the battery level. The software developmet based o the terrupter processes. Ths meas that most cotrol fuctos are fulflled the terrupter routes. The lowest process s the PWM cotrol part ad the upper oes chage the publc cotrol varables, whch make the PWM output s adjusted to cotrol both motor speeds. The software dagram s show fgure.. The team cotroller A Petum III PC s used as the cetral computer to cotrol the three robots va a sgle-drecto wreless trasmtter. A frame grabber gets the vdeo mages from the vdeo camera ad trasfers the mage data to the PC AM that wll be accessed by the program. The system programs eed to predct the chages of the ptch ad pla the path for each robot. Some mache learg programs for team strategy ad team formato are also eeded to w the game. Fgure 3 shows a block dagram of the robot cotrol system..3 Vso system hardware The vso system hardware cludes two parts. Oe s JVC TK-C38 vdeo camera that has the horzotal resoluto of 7 TV les ad provdes three kds of output vdeo format. Aother s the Ellps o frame grabber that s desged to capture dyamc mages up to at 5Hz ad at 5Hz. The dgtal mages are stored to the AM drectly stead of SVGA vdeo memory, whch make t easy to access. It also uses the PCI DMA mode to trasfer the data, whch meas the trasferrg rate ca be up to 3MB/s ad oly cost some CPU dle tme. The maufacturer provdes some developg lbrary uder Wdow 95/98 ad Wdow NT wth the hardware as well. Fgure obot software flowchart. Vso system software Sce the hardware maufacture has provded the basc lbrary MFC style, the developmet of the computer vso system s modular ad ca be mplemeted stages. The modules cosst of mage pre-processg, object detecto ad object trackg. The mage pre-processg module maly deals wth the whte ose ad makes the mages more clear to recogse. The object detecto module extracts the cocered object from the mages ad the object-trackg module provdes the posto ad search rego of each object. Usg predcato algorthm has two advatages for the whole system. Oe s t ca reduce the search rego greatly ad save much processg tme. The other oe s that t ca provde some predctg formato for the pathplag module. 3. Image pre-processg Commoly the processed mage s the format of GB mode ad the colour depth s bts. It meas the maxm colours ca be 6.7 mllo. Because of cosderable low resoluto, the whte ose mght reach hgh level. Ths module maly reduces the whte ose o the backgroud. Normally, the backgroud cludg the ptch ad others s very steady, so t s very useful for the followg stages to get several frames of backgroud ad average them. Because the whte

3 ose s radom, the mage qualty of the backgroud ca be mproved dramatcally. threshold for each colour ad H value s oly related wth the colour tself. The equato used by the coverter s showed as followg. I + G+ B; H arccos ( G) + (( G) + ( B) ) ( B)( G B) fb< G; H π arccos ( G) + (( G) + ( B) ) ( B)( G B) () Fgure 3 The robot cotrol system The mage-processg flter s very helpful may applcatos. I ths case, a hgh-pass flter s used to get the border of the ptch. Wth the hgh-pass processg, the border ca be much clearer to be recogsed. The lght o the ptch ca affect the later process greatly. If there s too much lght ad reflecto, the colour of the object s dffcult to recogse. The pre-processg module uses a auto-addtve adjustg to balace the lght. The total lumace o the ptch s frstly calculated, ad the coverted to a sutable lumace rate for each pxel. The program ca adjust the opto for the frame grabber f the rate s partcular rage.. Object detecto The object detecto module cossts of two ma parts. Oe s the colour flter ad aother s object posto. Colour flter works to get the mage oly wth cocered colours ad object posto s used to get the posto of each object.. Colour flter The FIA rules state that the team colours are blue ad yellow whle ball colour s orage. The rules also state that markers wth a uque colour ca be used to mark each robot dvdually. To posto the object, these colours must be detected ad extracted. Ths meas that t s ecessary to recogse two colours for robot detecto ad oe colour for the ball. Wth the colour flter, the smple mage wth Yes/No wll be avalable. Whle the mage GB format s ot sestve to colour, GB to HSI coverter s used. Image HSI format ca be set cosderably small fb> G; 3m(, G, B) S ; I Whle the rage of each colour s dfferet from each other, the threshold eeds to be dfferet as well.. Object detecto If the cocered parts of mage are avalable, t s very easy to posto the object. Whe all the pxels are from oe object, the posto wll be the average co-ordato. But there are always some separated pxels the mage. A quck ad smple way to remove the ose s to terate through the whole mage ad remove ay small groups of pxels. A process amed ope cludg the eroso ad dlato operatos ca be used to do so []. The eroso operato s a smple flter used to delete small groups of the same colour. It works by passg a flter template over each pxel the mage. If the flter completely matches a group of pxels the template, the the pxel at the cetre of the flter s coped to ew mage array. The dlato operato uses same flter template as the eroso operato, but t works the opposte way. The cetre pxel of the flter wll be coped to the ay coected pxels ew mage array. Wth eroso ad dlato operatos the ew mage ca be much clearer. The t s mportat to get all the coected pxels. Because some of cocered pxels mght be gored or the colour flter mght let some other pxels, a patter s used to do the recogto job. The smlarty s measured ad the object s foud f smlarty s greater tha the pre-set level.

4 5. Object trackg ad predcto The predctos for movemets of the ball ad the robots have two ams. Oe s to reduce the search regos of each object. Commoly the speed of the objects, robots ad ball, s less tha meter per secod. The maxmum dstace for each object to move ths short perod s less tha cm whe the operato frame rate s 5Hz. The search rego ca be oly.% of the whole ptch. Wthout the reducto of the search rego, the computato load s too heavy to bear. The secod am s the predctos ca provde some useful formato for path plag module ad strategy module. There are three kds of predcto methods mplemeted the applcato: postobased predctos, velocty-based predcatos, ad accelerato-based predctos [,3,6,8,]. 5. Posto based predcto Ths s a smple predctve trackg method. It s the oly useful method whe the object follows rregular movemet. The ext posto wll be the same as the prevous oe ad the search rego wll be the rectagle rego wth the sze greater tha the maxmum dstace that the object ca move. Ths method costs less o the computato, but the search rego s bgger tha other two methods. It ca ot provde valuable formato for the strategy module. The search rego ca be reduced to about.% of the whole ptch. 5. Velocty based predcto The velocty-based predct the ext posto usg several prevous postos. The method s based o the frst-order polyomal fttg o the assumpto that the varace of the object velocty s low, ad t moves a straght le f the object does ot ht the border. The search rego for the velocty-based predcto s much smaller tha that of posto based predcto ad t ca provde more useful formato tha the posto-based predctos. The predctos have two steps: oe s to fd a fttg straght le ad aother oe s to calculate the future posto. Whle the measured postos cota some errors from the real oes, the measured postos are ot oe straght le though the object ru a straght le. The the fttg straght le ca ot be calculated by usg oly two ear pots. If so, the errors of the robot oretato ca be cosderably bg. A good method s to accept the measurg error for each posto ad use more pots to ft a straght le. Assumg ow there are + pots ( x, y ) to x, ad straght le s ( x ) a + a x. The ( ) y p error from each pot to the straght le s the we have d [ P ( x ) y ], To make d least, a ad a should be the soluto of the followg equato ad s s x j a + s a s a + s a t t t y j x () It s sutable to let 5 the applcato. The ext step to calculate the future posto s: dx ( t) dx ( t ) V ( t) (3) dt ( t ) dx(t) ad dt(t) ca be approxmated to X(t) ad T(t): X(t ) X(t ) V(t ) () T(t ) T(t ) The equato for co-ordate predcto usg velocty s Xˆ (t) X(t ) + V(t) T(t) (5) By substtutg t equato 3, ths becomes: X( t ) Xˆ ( t ) X( t ) + T( t ) (6) T( t ) Note that the result should be projected oto the straght le that has bee calculated. 5.3 Accelerato based predcto Most of the object movemets the applcato are costraed by obstacles. It s very dffcult for objects rug wthout chagg ther speed. For example, the ball rus whle ts speed decreases utl ext httg. The usg the secod order polyomal fttg that presumes the varace of accelerate s chagg less ca be more sutable. The accelerato-based predctos ca ot reduce the search regos ay more tha the veloctybased predctos ad cost more computato tme.

5 But t provdes a accurate method for strategy module. I other applcatos, t ca be much more useful tha the other two methods. The calculato has two steps. Frst oe s to get the prevous trackg pots. For coveece, t ca be a straght le or d order polyomal curve. The secod step s to calculate the future posto. x to Assumg that there are + pots ( ), y ( ) y x, ad straght le ( x) a + a x + a x p The error from each pot to the straght le s The d [ P ( x ) y ] soluto of followg equato s a + s a + s a s a + s a + s a ad To make d least, a, a, a should be the s a 3 + s a 3 + s a s x j, t t t t I the applcato, t s sutable to let 5. y j x (7) The ext step to calculate the future posto s: X ( t) X ( t ) + V ( t ) dt dv ( t) dx ( t) A( t) ( ) ( t) + A( t) dx ( t ) dt ( t ) dx ( t) V ( t) These ca the be used to produce the fal accelerato equato: T(t) X(t) ˆ X(t ) + X(t ) T(t ) (9) T(t) T(t ) + X(t ) X(t ) T(t ) T(t ) Note that the result should be projected oto the straght le have bee calculated. 6. Expermet results (8) To evaluate the performace of the varous predcato algorthms, a ball was maually rolled across the ptch repeatedly. Each predcto algorthm was the used to predct the movemet of the ball. The frame rate for ths expermet was early 5 frames per secod ad the resoluto was Wth the fast CPU speed, all these algorthms could acheve a hgh frame rate. However whe the umber of the object creased, the frame rate of accelerato-based predctos was a bt lower tha other two. These fgures show the posto errors for the algorthms. There are two les each fgure descrbg the errors both X, Y drectos Fgure Errors of posto-based predctos Fgure 5 Errors of velocty-based predctos Fgure 6 Errors of accelerato-based predctos

6 7. Coclusos ad future work obot soccer s a challegg task that volves may areas of electrocs ad computer scece. The vsual trackg system s the vtal part of ts whole system. I fact the cotrol system program s embedded the vsual trackg system software ad takes the dle tme of the vsual system. The paper has preseted such a vsual trackg system for our Essex Wzards team. The trackg algorthm s able to accurately track multple objects at early 5 frames per secod. The system provdes a effectve foudato for further developg strategy ad path plag modules for the robot soccer competto. The ext stage of our research s to tegrate whole system from the robots, the PC-based vsual trackg system to the strategy ad path plag modules so that a workg platform ca be used for further mult-aget system research. efereces [] A. Agah ad K. Tae, obots Playg to W: Evolutoary Soccer Strateges, Proc. IEEE It. Cof. o obotcs ad Automato, Albuqerque, New Mexco, pp , Aprl 997. [] S. Blackma, Multple Target Trackg, Chapter Methods for Flterg ad Predcto, pages 9-, 986. [3] C. Brow, H. Durrat-Whyte, J. Leoard, B. ao, B. Steer, Cetralsed ad Decetralsed Kalma Flter Techques for Trackg, Navgato ad Cotrol, Dept. of Eg. Scece, Uv. of Oxford, May 99 [] A. Dews, A Vsually Based Trackg System for use a Game of obot Soccer, MSc dssertato, Departmet of Computer Scece, Uversty of eadg, Sept [5] A. Dogoul, A. Collot, Applyg a Aget- Oreted Methodology to the Desg of Artfcal Orgazatos: A Case Study obtc Soccer, Autoomous Agets ad Mult-Aget Systems, pages 3-9, 998. [6] C.S. Hog, S.M. Chu, J.S. Lee, K.S. Hog, A Vso-Guded Object Trackg Ad Predcto Algorthm for Soccer obots, Proc. IEEE It. Cof. o obotcs ad Automato, Albuquerque, New Mexco, pages 36 35, Aprl 997. [7] H. Hu, K. Kostasds, Z. Lu, Coordato ad Learg a team of Moble obots, Proc. of IASTED obotcs & Automato Cof., Sata Barbara, pp , Oct [8] K. Ha, M. Veloso, eactve Vsual Cotrol of Multple No-Holoomc obotc Agets, Carege Mello Uversty, 998. [9]. Ja, et al., Mache Vso, McGraw Hll, 995 [] J-H Km, H-S Shm, H-S Km, M-J. Jug, I-H Cho, J-O Km, A Cooperatve Mult-Aget System ad Its eal Tme Applcato to obot Soccer, Proc. IEEE It. Cof. o obotcs ad Automato, Albuquerque, New Mexco, pp , 997. [] H. Ktao, obocup: The obot World Cup Itatve. I proceedgs of the st It. Coferece o Autoomous Aget (Agets- 97)), Mara del ay, The ACM Press, 997. [] Z. Koroa, M. Kokar, Multsesor Itegrato the Trackg of Ladg Arcraft, Proc. 99 IEEE It. Cof. o Multsesor Fuso & Itegrato for Itellget Systems, Las Vegas, pp , 99. [3] E. Kruse, F.M.Wahl, Camera-based Observato of Obstacle Motos to Derve Statstcal Data for Moble obot Moto Plag, Proc. 998 IEEE It. Cof. o obotcs ad Automato, Leuve, Belgum, pages , May 998. [] S-W. Park, J-H. Km, Eu-Hee Km, Ju-Ho Oh, Developmet of a Mult-Aget System For obot Soccer Game, Proc. IEEE It. Cof. o obotcs ad Automato, Albuquerque, New Mexco, pages 66 63, Aprl 997. [5] W.M. She, J. Adb,. Adobbat, B. Cho, Buldg Itegrated Moble obots for Soccer Competto, Proc. IEEE It. Cof. o obotcs ad Automato, Leuve, Belgum, pages 63 68, May 998. [6] M. Veloso, P. Stoe, K. Ha, S. Achm, The CMUted-97 Small obot Team, Carege Mello Uversty, 998. [7] FIA Web ste: