POSTRACK: A Low Cost Real-Time Motion Tracking System for VR Application

Transcription

1 POSTRACK: A Low Cost Real-Te Moto Trackg Syste for VR Applcato Jaeyog Chug, Nagyu K, Gerard Joughyu K, ad Cha-Mo Park VR Laboratory, Departet of Coputer Scece ad Egeerg, Pohag Uversty of Scece ad Techology Sa 31, Hyoja-dog, Na-gu, Pohag, Kyug-buk, , Korea [email protected] Abstract Oe of the obstacles to the prolferato the VR techology to dgtal cotets s the expesve, trusve, cubersoe ad brttle ature of the sesors that are requred to detect user s tet. Whle optcal trackg, beg wreless wth respect to the user s body, has bee regarded as oe soluto to ths usablty aspect, the tradtoal probles of establshg arker correspodece ad resolvg ther occlusos real te stll rea. Oe aveue of efforts to address these probles s to sply add ore ad ore hardware or processg power, akg the trackg syste too expesve for geeral usage, whle aother ajor effort looks to drectly track hua body parts, but usually suffers fro the ablty to track pot features. I ths paper, we preset a relatvely expesve e.g. rus o a hgh-ed PC, but reasoably accurate real te optcal oto trackg syste, called the POSTRACK, that ca stll fd a wde rage of applcatos for VR. POSTRACK uses four cheap 8-bt grayscale caeras attached wth frared LED s, ad the user wears several 1~5 hghly reflectve arkers. The arkers are desged to be very easy to wear sap-o, ad eve fashoable. The four caeras are calbrated usg well kow algorths, ad the tal arker assgets are foud by a sple heurstc based o the oral hua posture, whle the fudaetal atrces are used to fd blob correspodece ad copute the 3D postos of the arkers. Sce the sap-o arkers are rather large ad ther postos are coputed fro ther age cetrods whch costatly chage, the coputed 3D posto data are jttery, ore so tha other trackers that use uch saller arker szes, or copared to agetc trackers. Whle the techology s old.e. uses stadard vso ad stereo algorths, the egeerg of the POSTRACK strkes a good balace betwee cost ad ts capablty, ad we deostrate ths through results fro usg t for gesture oto recogto ad oretato trackg for 3D potg. 1. Itroducto Oe of the obstacles to the prolferato the VR techology to dgtal cotets s the expesve, trusve, cubersoe ad brttle ature of the sesors that are requred to detect users tet. Whle optcal trackg, beg wreless wth respect to the user s body, has bee regarded as oe soluto to the usablty aspect, the tradtoal probles of establshg arker correspodece ad resolvg ther occlusos real te stll rea to be dffcult. Oe aveue of efforts to solvg ths proble s to eploy expesve hardware e.g. wreless agetc trackers, hgh resoluto caeras, etc. ador crease the coputg power, akg the trackg syste too expesve for geeral usage ad everyday applcato [1][11]. Aother ajor approach looks to elate the eed for the arkers, drectly trackg parts of the body, for stace, the hads, face, ad lbs, however, ths requres further processg for feature detecto ad usually suffers fro the ablty to track pot features [6][7].

2 We preset a relatvely expesve e.g. rus o a hgh-ed PC wth al hardware set-up, but reasoably accurate e.g. wth few ceteters optcal oto trackg syste, called the POSTRACK, that ca stll fd a wde rage of applcatos for VR. Whle the techology s old.e. uses stadard vso ad stereo algorth, we beleve that the egeerg of the proposed trackg syste strkes a good balace betwee cost ad ts capablty. We deostrate ths by applyg POSTRACK to two dfferet VR applcatos, aely, gesture oto recogto ad object selecto, ad evaluated ts utlty perforaceusablty vs. cost agast the popular but expesve agetc tracker. 2. Related Work: Vso based Trackg Lately, there has bee a rekdled terest the vso-based trackg for the ext geerato user terface. Vso-based trackg used to suffer fro the classcal probles of establshg arker or feature correspodeces ad occluso probles, whch ca partally be resolved real te by usg fast coputers. The ever-creasg coputg power of desktop coputers sees to be the ajor cause to ths revved terest. The ost popular 3D trackers used VR applcatos are the agetc ad ultrasoc types, both usually cubersoe to use for ot beg wreless wreless versos are uch ore expesve, ad oreover, relatvely stll too expesve to use for the everyday desktop applcato at least, several hudred to few hudred thousad dollars rage. Most coercal oto capture systes eploy vso-based ethods because they offer the wreless coveece, stead, requre hgh uber of specal hgh precso frared caeras ad heavy coputg power for accuracy ad speed eeded for the professoal aato producto, the a applcato area of oto capture [1][11]. Obvously, cost wse ad set-up wse, these systes are ot ft for geeral VR terfaces whch oly eed several trackg pots ad lower accuracy. Aother ajor approach looks to elate the eed for the arkers, drectly trackg parts of the body, for stace, the hads, face, ad lbs, however, ths requres further processg for feature detecto ad usually suffers fro the ablty to track pot or detaled features [6][7]. As both caeras ad coputers becoe cheaper copared to ther capabltes these days, ay augetedvrtual realty systes are startg to explot ther ow IR-based optcal trackg fraework at a reasoable cost, accuracy ad speed [1][2][3][5]. Our work shares the sae goal. 3. POSTRACK POSTRACK rus o a stadard PC wth a 8 MHz Petu III processor ad uses four cheap 8-bt grayscale caeras attached wth frared LED s. We opted to use four caeras to accout for possble arker occluso, yet keep the overall cost relatvely low could have used just two or ore tha four. For vdeo capturg tasks, we use PCI frae grabber that ca acqure four chael ages fro four NTSC aalog caeras at about 24 fraes per secod. For trackg purposes, the user wears oe or ore retro-reflectve arkers. As the arker does ot have ay oretato, oe arker trackg aouts to just posto trackg, that s, oretato trackg s possble by trackg ore tha two arkers by calculatg ther relatve postos. The arkers are ade of a ateral called the ScotchLte 1 wth about thousad tes hgher reflectace to lght tha everyday aterals. The arkers are also desged to be very easy to wear reflectve ateral wrapped aroud a flexble sap-o 1 ScotchLte s a regstered tradeark of the 3M Corporato.

3 etal bad, ad eve fashoable 5 arkers cost about $8. Fgure 1 shows the IR caera ad arkers. a b c Fg. 1. a Vew of a caera outed IR LED s ad a IR flter. b Retro-reflectve arkers upper s whe sapped, ad lower s whe opeed c A user wearg fve arkers at oes akles, wrsts ad belly. After calbratg the caeras descrbed the ext secto, the 2D ceters of gravty of the arkers the respectve age space are calculated. The atchg arkers betwee the four captured ages are solved for usg the eppolar costrat, the the 3D postos of the arkers are coputed. Whe usg ultple arkers, the arker assgets e.g. arker 1 s for the rght akle are ataed by usg a predcto-based algorth after a heurstc talzato at the begg of trackg Caera Calbrato The caera calbrato s carred out usg the calbrato fuctos developed by the Itel s Ope source coputer vso lbrares OpeCV [4]. OpeCV caera calbrato fuctos are used for calculatg trsc ad extrsc caera paraeters by havg the four caeras referece o kow vsual features, for exaple, vertces fro a black ad whte patter age. The caera calbrato process usually operates by uercal optzato, ad our case, the calbrato works well coverges to a good soluto, as the approxate locato ad oretato of the caeras are already kow at top four corers of a rectagular volue, see Fgure 2. Slght odfcatos were ade to the OpeCV calbrato fuctos to ake t work for ultple caeras ad black ad whte ages Marker Trackg Whe lt by the frared LED s, the arkers the four ages obtaed by the capture board appear as whte blobs, therefore, after perforg a sple thesholdg operato e.g. flter out every pxel wth gray scale hgher tha about 248, 255 beg the axu, oly arker ages are left. A eda flter s appled to reove ay scattered ose. Fgure 3 shows two arkers at the user s wrsts see fro the four caeras before thesholdg. Usg the eppolar costrat that states that the correspodg arker the other age ust lke soewhere the eppolar le eppolar le for oe arker s gree ad the other s red, we ca fd the atchg arkers by solvg the followg equato that represets the eppolar costrat [12].

4 p l Fp r,where F s called the fudaetal atrx ad ca be obtaed by M -T r EM -1 l. p l s the posto of the arker oe age ad p r s the posto of the sae arker the other age the pxel coordates. M r ad M l are the atrces of the trsc paraeters of the two caeras. E s called the essetal atrx that establshes the lk betwee the eppolar costrat ad the extrsc paraeters of the stereo syste. Sce POSTRACK uses four caeras, we cosder caddate age pars that produce the least value for the above equato. Ifrared Caera a b Fg. 2: a Four caeras outed o the celg two frot caeras are show, the other two are the rear syetrc postos. b The graphcal vew of the calbrato process result. Fg. 3. Eppolar les for two arkers o the user s wrsts. The tal arker assgets are foud by a sple heurstc based o the oral hua posture, that s, assued fro a kow tal pose e.g. blob the lower left porto the age s for the left foot. 4. Applcato 1: Gesture Moto Recogto WeappledPOSTRACKasaputtoagestureotorecogtoodulefora terface to avgatg 3D vrtual evroet. Ths s perhaps the splest applcato of POSTRACK as t oly tracks oe arker placed o the wrst of the user. The user akes cotuous otos by ovg oe s ar, ad the postos of the arker tracked by POSTRACK are used as a put to the gesture recogto odule. Ths s a typcal deostrato of the VR techology that corporates ore atural teracto for hgher usablty ad a stulatg experece. We odeled four basc oto gesture for avgatg a 3D evroet: forward, stop, tur left, adtur rght. The oto gestures are show Fgure 4.

5 stop forward left rght Fg. 4. Moto gestures for VE avgato ad the trajectores of the four 3D oto gestures Gesture Recogto Asde fro just recogzg the gesture tself, ovg gestures creates aother subproble, that s, detectg the startg ad edg pots of the teded gesture the dst of posto data that are streag. Oe sple soluto s defe a stll state, ad for stace, requre the user to be statoary for few secods to sgal the start ad the ed of a oto coad. To overcoe such coveece, our ethod looks for a eagful oto patter fro a strea of data cotaed wth a fte data search wdow. The data search wdow starts at a u legth e.g..25 secods, or 5 fraes at 2 Hz saplg rate fro the curret frae, ad grows to a predefed axu e.g. 3 secods, or 6 fraes at 2 Hz saple rate. The predefed u ad axu legths of the search wdow are detered based o a heurstc that a gve gesture coad would requre at least that u aout of te to be carred out, ad ust ot exceed that axu aout of te to be copleted. The varyg legth of the oto coad s hadled through a oralzato process of the sapled data. That s, the oto coad teplate data sze s ether trucated or elogated to ft the put data sze before applyg the atch algorth. Thus, as far as the durato of the oto coad s kept wth a reasoable boud, t wll be recogzed. The above search ad atch process s repeated at every data saplg perod whch s about 2 Hz. Oce a gesture s recogzed the data ca be further aalyzed for addtoal put propertes such as speed or accelerato. However, beg te depedet, ths sple algorth ca ot hadle gestures that are slar part, for stace, betwee a C ad a O oto. A C gesture would be recogzed the dst of gvg a O gesture. Gve a data search wdow, a partcular oto gesture s foud through a correlato based atch algorth. The correlato aalyss bascally aalyzes for how well a regresso fts the sapled data. The forula show Fgure 6 coputes for a easure of the qualty of the ft betwee the put ad the teplate oto data, ad the correlato coeffcet s coputed the x, y, ad z desos. If the correlato coeffcet s hgher tha a predefed threshold value e.g..85, +1 represetg a perfect correlato, a atch s deeed to occur. By usg the correlato aalyss, the correct classfcato rate s ade

6 u I u I T T u I T u I T u Corr where x, y, z u curret te dex T Teplate data, I Iput data uch less sestve to slght trackg error. I addto, the recogto s ade depedet fro the sze or locato of the gesture, thus there s o eed for data oralzato. Fg. 5. Searchg for a atch the data search wdow. Fg. 6. Coputg for the correlato coeffcet Coparatve Syste Perforace We tabulated ad copared the gesture recogto rate betwee whe usg the POSTRACKadwheusgtheFASTRAK 2 agetc tracker. The put were captured sultaeously.e. the user wore the arker ad were attached wth the agetc tracker recever at the sae te for a far coparso. Fgure 7 shows the coparso result ad there s o sgfcat dfferece the recogto rate. Gesture recogto, by ts algorthc ature, s qute sulated fro sall trackg errors, ad thus a syste lke POSTRACK proves to be a suffcet, yet cost effectve data acqusto syste for VR. 2 FASTRAK s a regstered tradeark of Polheus, Ic.

7 Tur Tur rght Go Stop Fg. 7. The coparatve recogto rate betwee POSTRACK ad FASTRAK. 5. Applcato 2: Object Selecto by 3D Potg As a bt ore challegg applcato, we appled POSTRACK to 3D potg for object selecto vrtual evroets. 3D potg, also kow as the vrtual poter etaphor or ray castg [9], s oe of the ost popular teracto techques VR. To extract the oretato of the ray, two arkers are tracked, oe o the wrst See Fgure 8 ad oe placed o a swtchg devce the swtch s eeded to cofr the selecto after potg D Cursor Posto Calculato The drecto of the ray s coputed sply by coputg for the drecto vector fro the two arker postos. Object selecto by ray castg s fact a 2D teracto techque because the selecto s ade o the age plae at whch the 3D object ad the ed of the ray are projected to. The posto of the cursor or the ed of the ray s sply the posto o the age plae where the vrtual ray tersects wth t. Iagecursor plae had drecto curret cursor pot Fg. 8. Vrtual potg ad the arker placeet.

8 5.2. Coparatve Syste Perforace Aga, we copared the user perceved syste perforace by rug a selecto test usg the POSTRACK ad the FASTRAK agetc tracker. We had subjects ake potg selectos to a radoly geerated sx objects ad easured the object selecto te. The sae tests were repeated for fve dfferet object szes. Fgure 9 shows the result of the coparatve perforace test. The test was coducted o a 24c by 18c projectve dsplay scree wth 8 x 6 pxel resoluto. The user was about 28c away fro the dsplay scree. The sze of the level 1 object was at 2 x 15 pxels, ad 6 x 55 pxels for the level 5. The results show that the selecto perforace starts to degrade wth level 2 objects about 15c wde. Ths s coparable to a desktop dsplay stuato where user s located about 4 c away fro the otor lookg at a 3 c wde object or bg co. For ost VR applcatos, such accuracy s suffcet for all practcal purposes. Moreover, the jttery optcally tracked data due to the costatly chagg cetrod of the rregularly shaped blobs ca be soothed for better perforace. TeSec Fg. 9. The coparatve selecto te betwee POSTRACK ad FASTRAK. 6. Other Applcatos: Moto Evaluato for Trag ad Aato We have appled POSTRACK to other applcatos, although dd ot copare ts perforace agast the agetc tracker. POSTRACK s used o our VR-based otodace trag syste s called the Just Follow Me JFM [8]. JFM uses a vsual terface called the Ghost etaphor whch the oto of the traer s vsualzed real te as a ghost ovg out of traee's body. The traee, who sees the oto fro a chose vewpot, s to follow the ghostly aster as close as possble both tg wse ad posto wse. As a oto trag syste, t s requred to capture the oto of the user ad copare t to that of the referece oto data. Ideally, we would have to track all the jots the user s body, however, we oly track fve body pots, the wrsts, akles ad the belly, whch s suffcet to evaluate the closeess betwee two hua body otos. Lke ay VR applcatos, the oto evaluato schee of JFM requres oly approxate oto data, sce t s certaly ot ecessary for the user to reproduce the aster s oto exactly, but oly wth a certa tolerable boud. However, tg of postures s a portat ad plct evaluato crtero, thus a far ~15 Hz saplg rate s requred of the trackg syste. POSTRACK ca track fve arkers at about 15 Hz.

9 POSTRACK s also used for real te aato, usg verse keatcs, to cotrol lbs of a avatar usg a sall uber of sesors or arkers. Aga, a saplg rate of at about 15 Hz s requred to produce a atural lookg aato. Fg. 1. Just Follow Me: the dace verso. As the user tres to tate a character the scree, oe s oto s captured ad copared to the referece oto. 6. Cocluso I ths paper, we have preseted POSTRACK, a low cost less tha $4 excludg the PC real te optcal oto trackg syste that perfors up to about 15 Hz saplg rate for fve arkers. As llustrated wth the uber of applcatos troduced ths paper, ay VR applcatos do ot requre p or pxel potg accuracy, but ay stll, requre a saplg rate of at least about 15 Hz so that that tracked object ay be redered soothly. We beleve that the egeerg of the POSTRACK strkes a good balace betwee cost ad ts capablty. The popular wred agetc trackers are ot oly cubersoe to use, but also sestve to etallc objects, ad ofte requre custo calbrato process. A syste lke POSTRACK ca ope doors to creatg ore stulatg terfaces to ay exstg applcatos ad brg VR out to the geeral publc. 7. Refereces [1] Madrtsch F., Gervautz M., CCD-caera based optcal beaco trackg for vrtual ad augeted realty, Euro-graphcs, vol. 15, o. 3, pp , 1996 [2] K. Dorfuller, Robust trackg for augeted realty usg retro-reflectve arkers, Coputers & Graphcs, vol. 23, o. 6, pp , 1999 [3] Mguel Rbo, Axel Pz, Ato L. Fuhra, A New Optcal Trackg Syste for Vrtual ad Augeted Realty Applcatos, IEEE Istruetato ad Measureet Techology Coferece, Budapest, Hugary, May 21 23, 21 [4] Itel, Ope Source Coputer Vso Lbrary Referece Maual, 21 [5] Chag C., Tsa W., Vso based Trackg ad Iterpretato of Hua Leg Moveet for Vrtual Realty Applcatos, IEEE Tras. O Crcuts ad Systes for Vdeo Techology, Vol. 11, No. 1, 21

10 [6] Fujyosh H., Lpto A., Real Te Hua Moto Aalyss by Iage Skeletozato, Proc. of IEEE Itl. Cof. o the Face ad Gesture Aalyss, 1998 [7] Wre C., et al. Pfder: Real Te Trackg of Hua Body, IEEE Tras. o Patter Aalyss ad Mache Itellgece, Vol. 197, pp , 1997 [8] Yag, U., Just Follow Me: A VR based Moto Trag Syste, Eergg Techologes, ACM SIGGRAPH, 21 [9] Poupytrev I., Weghorst S., Egocetrc Object Mapulato Vrtual Evroets: Eprcal Evaluato of Iteracto Techques, Coputer Graphcs Foru, pp , EUROGRAPHICS, 1998 [1] Vco, 21 [11] MotoAalyss, 21 [12] Trucco E., Verr A., Itroductory Techques for 3-D Coputer Vso, Pretce Hall, Ackowledgeet The work preseted ths paper has bee supported part by the Korea Mstry of Educato s BK21 Project ad Korea Scece ad Egeerg Foudato KOSEF, ad separately by the KOSEF supported Vrtual Realty Research Ceter.