A Multi-Camera System on PC-Cluster for Real-time 3-D Tracking

Transcription

1 The 23 rd Conference of the Mechancal Engneerng Network of Thaland November 4 7, 2009, Chang Ma A Mult-Camera System on PC-Cluster for Real-tme 3-D Trackng Vboon Sangveraphunsr*, Krtsana Uttamang, and Pongsakon Pedpunsr Robotcs and Advanced Manufacturng Lab, Department of Mechancal Engneerng, Chulalongkorn Unversty 254 Phyatha, Patumwan Bangkok, Tel. +66 (0) , +66 (0) , Fax. +66 (0) *E-mal: [email protected] Abstract In ths research, a real-tme mult-camera system has been developed for trackng a movng object n 3-D space. Because of a large amount of data obtaned from multple cameras, a PC-cluster system, a group of lnked computers, has been mplemented to reduce computatonal load n one computer and make real-tme trackng possble. The system conssts of multple cameras and a PCcluster, four cameras and two PCs n ths research. CCD camera s connected to computer va IEEE nterface. Synchronzaton mechansm has been developed usng PCI-to-PCI data movers wth fber optc connecton. After camera calbraton process, a 3-D poston of movng object s calculated from a set of correspondng 2-D mage coordnates of detected object feature usng lnear trangulaton method. The system has been tested by trackng a sphercal object wth moton generated by Mtsubsh PA10 robotc arm. The experment results show that the system can track the object n real-tme wth acceptable accuracy both poston and velocty. As a prototypcal applcaton n reverse engneerng, the developed system has been shown to be a hybrd vson/touch-probe measurng machne when movng the target object on surface needed to be 3-D reconstructed. Keywords: multple cameras system, PC-cluster, vson-based trackng, 3-D pose estmaton 1. Introducton Mult-camera system has become an actve research topc n computer vson because of ts capablty to deal wth occluson. As the number of camera ncreased, a large amount of data needs to be processed. Ths affects realtme performance of such system. One approach to overcome ths problem s usng multple connected computers, called PC-cluster, to reduce computatonal load n one computer such as [1-3]. Another approach n [4] uses hardware embedded n each camera as local processng unt to process data before sendng result to man processor.

2 In ths paper, a mult-camera system based on PC-cluster has been developed. The connecton between PCs has been made usng PCI-to-PCI data mover card va fber optc cable. To develop as a trackng system, fve processng modules have been desgned and arranged n each PC accordng to system archtecture. The system can track an object, a sphercal object n ths research, n 3-D space n real-tme. The detal wll be descrbed n the followng sectons. 2. Overvew D trackng process To perform 3-D trackng, many processes have been nvolved such as camera calbraton, feature extracton, and 3-D pose estmaton. The flow of the processes s as follows: 1) Calbrate cameras. 2) Capture mage from each camera synchronously. 3) Extract object feature n each mage. 4) Estmate 3-D pose of object usng extracted feature coordnates from step 3. 5) Dsplay result. 6) Repeat step 2-5 for each frame Processng modules The trackng process descrbed above has been desgned as fve processng modules as follows: Camera calbraton module Image capturng module 2-D mage processng module 3-D pose estmaton module PC to PC communcaton module Fg. 1 The arrangement of the processng modules on PC-cluster The arrangement of the processng modules n 3-D trackng process s shown n Fg.1. The PC to PC communcaton module embedded n each PC for transferrng data between computers has not been shown n the fgure. Before usng the trackng system, all cameras need to be calbrated wth camera calbraton module whch located n the man PC when mplementng the system. 3. System Implementaton The PC-cluster system n ths research has been mplemented wth two computers, one s local or man computer and another s remote computer. Local computer controls overall processes both contaned n t and n remote computer. 3.1 Hardware The two PCs of the system consst of the man PC whch has Pentum IV 3.2 GHz as processor and 2GB of RAM nstalled and the remote PC whch uses Pentum IV 2.0 GHz wth 512 MB of RAM. They are connected to each other va fber optc cable wth datablizzard PCI-to-PCI data mover card. Each PC has 2 CCD cameras, PxeLINK PL-A741-BL, wth 16mm lens connected through IEEE-1394 ports. The system has been shown n Fg. 2.

3 Fg. 2 The developed system wth four cameras and two computers 3.2 Software The archtecture of cluster system conssts of two types of applcaton. Control applcaton operates on local computer and servce applcaton operates on remote computer. Its schematc dagram s shown n Fg.3. Fg. 3 The software archtecture of PC-cluster Servce applcaton Servce applcaton s a Wndows-based applcaton whch runs n the background of Wndows operatng system. In each PC, a servce applcaton wll be responsble for mage capturng and 2-D mage processng, so an mage capturng module and a 2-D mage processng module are contaned n t. Moreover, the servce applcaton has been developed to perform communcaton between PCs n PC-cluster system va PCI-to-PCI data mover card. It also provdes nterfaces to other Wndows applcaton Control applcaton For the developed system, one PC has been used as man PC. It controls remote PCs for trackng the target object. In the man PC, control applcaton has been developed on the top of servce applcaton descrbed earler by usng Component Object Model (COM) archtecture. The control applcaton receves 2-D mage coordnates of the target from servce applcaton and estmates 3-D poston of the target n the world coordnates. So, the 3-D pose estmaton module s contaned n ths applcaton. Moreover, the control applcaton also provdes applcaton user nterfaces. User can control trackng system through ths applcaton. The archtecture of the PC-cluster wth processng modules descrbed n secton 2.2 s shown n Fg.4. The PC to PC communcaton module s mplemented usng message passng. It wraps data nto programmng data structure called message and pushes t to message queue to watng for processng. The local computer contans the camera calbraton module and the 3-D pose estmaton module n addton to the three modules for remote computer.

4 Fg. 4 The archtecture of PC-cluster wth processng modules The applcaton software has been mplemented usng Mcrosoft Vsual Studo 2005 wth C++ language on Wndows XP SP2 operatng system. The control applcaton can dsplay trackng results n real-tme on graphcal screen. OpenGL s used as graphcs lbrary. The control applcaton s shown n Fg.5. calbratng camera wth ts pros and cons. In ths research, two methods have been mplemented, Tsa s [7] and Zhang s [8] method. For Zhang s method, radal dstorton model has been modfed to reduce computatonal tme n reconstructon process, see [9] for more detals. Chessboard calbraton pattern used for calbraton process has been desgn as 8x8 rectangles wth 20 mm n wdth and heght as shown n Fg. 6. The 49-pont correspondences have been used n order to compute camera matrx. Fg. 6 The chessboard pattern for calbraton Fg. 5 The software mplementaton wth graphcal user nterface 4. System Operatons 4.1 Camera calbraton Camera calbraton s the process of estmatng of ntrnsc and extrnsc parameters of camera model whch s a matrx mappng between world and mage coordnates [5, 6]. These parameters are mportant for 3-D reconstructon. There are many methods for Crcular mark n the calbraton pattern s used for automatc dentfcaton of pattern orentaton n the captured mage. 4.2 Feature extracton The accurate detecton of mage features s requred n applcatons of 3-D reconstructon. In ths paper, the sphercal object whose mage always crcle has been used as trackng target. The crcular Hough transform has been employed to detect center pont and radus of the object. 4.3 Pose estmaton Consder the multple cameras system whch has n cameras, gven P s the -th camera matrx, and x s pont n the -th mage of the 3- D world pont X and correspondng to camera

5 matrx P, then we have x relaton of vector cross product: 0 Then PX. By usng x PX (1) where x y x p Xp X 2T p Xp X 2T p X y p X (2) jt p s the j-th row of camera matrx P From each camera, only 2 equatons of (2) are ndependent, let choose the frst two equatons from each camera. A matrx L can be obtaned by stackng up equaton (2) as n [5, 10]: x1p1 p 1 2T y1p1 p1 x2p2 p 2 2T LX y2p2 p2 X 0 xnpn pn 2T y npn pn (3) Because of nose embedded n the captured data, we cannot obtan the exact soluton of the above equaton. The soluton for X can be obtaned by least-square soluton (LS) of equaton (3). LS gves an accurate result but t has no geometrcal meanng. The other method s the bundle adjustment wth Levenberg-Marquardt optmzaton (LM). Ths method tres to mnmze geometrc mage dstance between measured mage pont and reprojected mage pont of the estmated 3-D pont. LM gves a better accuracy soluton but slower. Ths s because LM s an teraton-based method. In contrast, LS s faster than LM but the error s also more than LM. Both methods have been mplemented n ths research. Fg. 7 3-D pose estmaton process Before performng 3-D pose estmaton, all cameras have to be calbrated and mage capturng module need to be set for synchronous mage capturng. One camera has been chosen as prmary camera to send hardware trgger to other cameras, secondary cameras. In 3-D pose estmaton process, the 3-D pose estmaton module n control applcaton performs operatons as shown n Fg.7. 1) 3-D pose estmaton module sends software type trgger sgnal to local mage capturng module. 2) Local mage capturng module sends software trgger to prmary camera only. 3) Prmary camera sends hardware trgger sgnal to other cameras. 4) Both prmary and secondary cameras capture mages and send them back to mage capturng module whch attaches to them. 5) Image capturng module passes the captured mages to 2-D mage processng module to fnd 2-D mage poston of object.

6 6) In local computer, 2-D mage processng module sends 2-D mage postons to 3-D pose estmaton module. 7) In remote computer, 2-D mage processng module sends 2-D mage postons to PC to PC communcaton module. 8) Remote PC to PC communcaton module transforms 2-D mage postons nto transferrng message and transfers t to local PC to PC communcaton module. 9) Local PC to PC communcaton module receves transferrng message and transform t back to 2-D mage postons. Then 2-D mage postons are sent to 3-D pose estmaton module. 5. Experments D trackng The developed system can detect the movng target, whch s a sphercal ball, attached to Mtsubsh PA10 robot arm as shown n Fg. 8. The moton for the robot arm has been generated and the system tracks the movng target. The results have been shown n Fg Fg. 9, 10, and 11 show the comparson of the result from the measurements and the robot arm move n XY-plane, YZ-plane, and ZXplane, respectvely. The speed of the robot arm s set to 0.05 rad/sec. Fg 11, 12, and 13 show the system detectng the target movng helcally n Z- drecton, Y-drecton, and X-drecton, respectvely wth the same velocty. Fg. 9 The target moves n the XY-plane Fg. 10 The target moves n the YZ-plane Fg. 11 The target moves n the ZX-plane Fg. 8 The robot arm holds a sphercal target

7 and trackng n a well-defned envronment to reduce exposure tme of the camera. Fg. 12 Helcal moton n z-drecton Fg. 15 Lnear moton wth s-curve velocty profle Fg. 13 Helcal moton n X-drecton 5.2 Applcaton for reverse engneerng The developed system can be used as a 3-D coordnate measurng machne for reverse engneerng applcaton. In Fg. 16, the sphercal target moves on part that surface need to be reconstructed lke a probe. The trangular mesh created from the measurement data has been shown n Fg. 17. Fg. 14 Helcal moton n Y-drecton The target object moton wth s-curve velocty profle has been tested as shown n Fg. 15. The robot arm s programmed to move n y- drecton wth acceleraton and deceleraton set equal to 20 mm/s 2. The total dstant s 300 mm wth 40 mm/s constant velocty. The result shows that the system can track the target very well. We can mprove the result by usng faster cameras Fg. 16 The sphercal target movng on surface Fg. 17 The trangular mesh created from the measurement ponts

8 6. Concluson In ths paper, a real-tme 3-D trackng system usng multple cameras has been mplemented on PC-cluster. Two types of applcaton, control and servce applcaton, wth fve processng modules have been desgned and mplemented to each PC. The system can track a sphercal object moved by Mtsubsh PA-10 manpulator wth acceptable accuracy. The better result can be obtaned f usng better cameras. To apply for reverse engneerng task, the system can track the sphercal object movng on target surface. Wth the touch probe mprovement, the system can be a vson/touch-probe measurng machne wth flexblty and scalablty. 7. References [1] Arta, D., Yonemoto, S., and Tanguch, R. (2000). Real-Tme Computer Vson on PC- Cluster and Its Applcaton to Real-Tme Moton Capture. 5th IEEE Internatonal Workshop on Computer Archtectures for Machne Percepton, pp [2] Yoshmoto, H., Date, N., and Yonemoto, S. (2003). Vson-Based Real-Tme Moton Capture System usng Multple Cameras. Proceedngs of IEEE Internatonal Conference on Multsensor Fuson and Integraton for Intellgent Systems, 30 July-1 August 2003, pp [3] Kurhara, K., Hoshno, S., Yamane, K. and Nakamura, Y. (2002). Optcal Moton Capture System wth Pan-Tlt Camera Trackng and Realtme Data Processng. Proceedngs of the IEEE Internatonal Conference on Robotcs and Automaton, 2: pp [4] Garca, R., Batlle, J. and Salv, J. (2002). A New Approach to Pose Detecton usng a Trnocular Stereovson System, Real-Tme Imagng, pp [5] Hartley, R., and Zsserman, A. (2003). Multple Vew Geometry n Computer Vson, 2 nd edton, Cambrdge Unversty Press, Cambrdge, UK. [6] Trucco, E. and Verr, A. (1998). Introductory Technques for 3-D Computer Vson, Prentce-Hall, New Jersey, USA. [7] Tsa, R. (1987). A Versatle Camera Calbraton Technque for Hgh Accuracy 3D Machne Vson Metrology usng Off-The-Shelf TV Cameras and Lenses, IEEE Journal of Robotcs and Automaton, vol. 3. no.4, pp [8] Zhang, Z. (2000). A Flexble New Technque for Camera Calbraton, IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 22. no. 11, pp [9] Uttamang, K., and Sangveraphunsr, V. (2007). Mult-Camera System usng PC-Cluster for Real-Tme 3-D Pose Estmaton, Thammasat Int. J. Sc. Tech. 12, Aprl-June 2007, pp [10] Faugeras, O. and Luong, Q. (2001). The Geometry of Multple Images, MIT Press, Cambrdge, MA.