where p is the centroid of the neighbors of p. Consider the eigenvector problem

Transcription

1 Vrtual avgato of teror structures by ldar Yogja X a, Xaolg L a, Ye Dua a, Norbert Maerz b a Uversty of Mssour at Columba b Mssour Uversty of Scece ad Techology ABSTRACT I ths project, we propose to develop a prototype system that ca automatcally recostruct 3D scees of the teror of a buldg, cave or other structure usg groud-based LIDAR scag techology. We develop a user-fredly real-tme vsualzato software package that wll allow the users to teractvely vsualze, avgate ad walk through the room from dfferet vew agles, zoom ad out, etc. Keywords: LIDAR, Vrtual avgato, vsualzato. INTRODUCTION I ths project, we propose to develop a prototype system that ca automatcally recostruct 3D scees of the teror of a buldg, cave or other structure usg groud-based LIDAR scag techology. We develop a user-fredly real-tme vsualzato software package that wll allow the users to teractvely vsualze, avgate ad walk through the room from dfferet vew agles, zoom ad out, etc. To meet the challeges of occlusos, large gaps/holes, ad sgfcat ose LIDAR data, we employ ad develop robust, effcet algorthms for all major compoets of the system (Fgure ) cludg robust model estmato based o Radom Sample Cosesus (RANSAC) ad Prcpal Compoet Aalyss (PCA); fast ad effcet teror object classfcato/recogto; ad robust 3D recostructo algorthm. Fgure : System flow chart.. Herarchcal LIDAR data segmetato Gve the LIDAR data of a teror room (Fgure (a)), we wll coduct herarchcal segmetato ad groupg. More specfcally, we wll frst detfy all the major plaar regos such as floor, celg ad vertcal walls of the room (Fgure (b)). Next, we wll detfy ad extract dvdual objects such as chars, tables, etc that are lyg/attachg to floors/celg/walls (Fgure (c)). Fally, we wll further segmet each extracted object such as char to homogeous clusters (Fgure (d)), whch wll the be used the subsequet local shape prmtve fttg ad classfcato process... Idetfcato of floor, celg ad walls We wll frst compute the boudg box that ecloses the LIDAR data. Assumg that the LIDAR data s scaed from oe sde of the boudg box such as the wdows of the room, we wll the search for major plaar regos such as floor, celg ad walls the eghborhood of the other fve sdes of the boudg box. More specfcally, for each sde of the boudg box, we wll frst extract all the pot clouds that are wth a certa dstace of the curret sde of boudg box. We wll the use RANSAC [4] to fd the best plae that ft these selected pot clouds. RANSAC s a

2 hypothess geerato ad testg algorthm that s very robust for outlers. The ma dea behd the techque s to use a mmal umber of data pots eeded to estmate the model (.e. ftted plae), ad the cout how may of the remag data pots are compatble wth the estmated model, the sese of fallg wth a chose threshold. The plae fttg wll be based o the robust PCA algorthm whch ca compute local surface propertes based o local eghborhoods of sample pots. We wll fd the k-earest eghbors of a sample pot p, deoted by the dex set N p, The local surface propertes of the pot clouds ca be effcetly estmated by the egeaalyss of the covarace matrx C of a local eghborhood at sample pot p: where p s the cetrod of the eghbors of p. Cosder the egevector problem Sce C s symmetrc ad postve sem-defte, all ts three egevalues are real-valued ad the egevectors v l form a orthogoal frame, correspodg to the prcpal compoets of the pot set. Thus v approxmates the surface ormal at p, or other words, v ad v spa the taget plae at p. Note that s the sze of the eghborhood, whch serves as the scale-cotrol parameter, ad s depedet o the laser scaer resoluto,.e. how dese the pot clouds data wll be. For geerc teror scees, the above sx-sded rectagular cube assumpto of the teror scee wll ot always hold. Thus ths case, we mplemeted a ew rego-growg based segmetato algorthm. The basc dea of the regogrowg algorthm s: start wth a uvsted pot p, teratvely clude ts eghborg pot q, f the dstace betwee p ad q s smaller tha a threshold (e.g. the samplg desty), ad the dfferece betwee the ormal at p ad the ormal at q s also smaller tha a threshold ( degrees our case). Ths ca be doe a breadth-frst search, ad stop whe the above crtera s o loger hold. The the algorthm move o to the ext uvsted pot, utl all the pots are vsted. To speed up the computato, we employed volumetrc grd to store the pots, thus the coectvty ferece ca be doe statly by grd dexg. The surface ormal s estmated by the aforemetoed Prcpal Compoet Aalyss (PCA). After the rego growg, we further merge clusters that are co-plaar,.e. we would lke to merge all the clusters that belog to a bgger plae eve though they are ot coected. More specfcally, a cluster s merged wth aother f the ormal of both clusters are wth degrees of each other, AND the vector potg from the frst cluster's mdpot to the secod cluster's mdpot s more tha 85 degrees apart from the average of the two cluster's ormals (by the frst codto, these two ormals wll be already early detcal)... Idetfcato of dvdual objects ad extracto Oce all the pot clouds these major plaar regos such as floor, celg ad walls are detfed ad removed, there are suffcet separato betwee pots of dvdual objects such as chars, tables, etc that are lyg/attachg to the groud, celg, ad walls. We the proceed to group all these pots based o ther proxmty to each other by fdg coected compoets such that each pot a coected compoet s wth a gve dstace to at least oe more pot that compoet. Hece, all the pots belogg to a dvdual object such as a char would le a sgle coected compoet. We employed effcet grd-based rage-fdg algorthms by usg a volumetrc grd to store the pots to speed up the coected compoet aalyss process...3 Segmetato of dvdual objects to homogeous clusters Oce we segmet the whole LIDAR data to dvdual coected compoets whch may correspod to objects such as chars, tables, fas, etc, we wll proceed to further segmet (.e. dvde) each objects to homogeous patches/clusters based o some smlarty metrcs (e.g. dstace, ormal, curvature, etc). The segmeted homogeous clusters wll the be used the subsequet local shape prmtve fttg process. I ths project, we employed a rego-growg based segmetato algorthm. Ths s the same method we used for user-guded sem-automatc plaes detfcato for

3 geerc teror scees, descrbed Secto... The rego-growg based segmetato algorthm works very well our expermets...4 Statstcs based local shape prmtve fttg Oce we segmet the whole LIDAR data to dvdual coected compoets whch may correspod to objects such as chars, tables, fas, etc, f eeded, we ca proceed to further segmet (.e. dvde) each objects to homogeous patches/clusters based o some smlarty metrcs (e.g. dstace, ormal, curvature, etc). We frst segmet the object to dvdual clusters based o the same aforemetoed rego growg algorthm for each dvdual objects. We the dvde each segmet to quadratc patches. More specfcally, for each segmet: a. Ft quadratc to the curret segmet (use RANSAC method to oly ft lers) b. Calculate the error per pot the segmet (dstace from pot to quadratc) c. If the average error for the etre segmet s greater tha +/-%. Splt the segmet to 8 equal segmets. Repeat step a. The followg are the detaled mplemetato:. Gve a segmeted cluster, we compute ts local coordate system wth local org c by coductg Prcpal Compoet Aalyss (PCA) of all the pots ths cluster.. Trasform all the pots ths cluster to the local coordate system. 3. Wth the curret local coordate system, ft a local bvarate quadratc polyomal fucto Q(u, v, w) = w (Au*u + Bu*v + Cv*v + Du + Ev + F) of all the pots the cluster, by coductg weghted least square optmzato to mmze: Q( p ) ( p c ), where p are pots the cluster, ad s a oegatve, smooth, radal, mootoously decreasg fucto, usually wth fte support, e.g. a trucated Gaussa fucto. Sce the pots may cota outlers such as ose or artfacts, we further employed Radom Sample Cosesus (RANSAC) algorthm to tegrate wth the above quadratc fttg, so that the algorthm wll be robust to ose. RANSAC s a hypothess geerato ad testg algorthm that s very robust for outlers. The ma dea behd the techque s to use a mmal umber of data pots eeded to estmate the model (.e. quadratc fucto), ad the cout how may of the remag data pots are compatble wth the estmated model, the sese of fallg wth a chose threshold (we use the average fttg error evaluated from the quadratc fucto Q).. Progressve surface recostructo Sce the orgal LIDAR data ca be qute large, thus ths project, we developed a progressve surface recostructo algorthm. More specfcally, we wll utlze the result from the prevous herarchcal segmetato, ad progressvely trasmt the data to the surface recostructo algorthm cluster by cluster. There wll be two major types of data, oe s the plaar data, oe s the o-plaar object data. To hadle extreme large data, we further exted the aforemetoed data segmetato algorthm to two herarchcal steps. I the frst step, we coduct grd-based data segmetato usg a subset of the orgal data. For example, we sequetally subsample the data (e.g. %), ad coduct clusterg as descrbed before. We the label the cells the embeddg grd by the clusters. I the ext step, we load the whole set of the orgal data sequetally, ad label each data pots based o the label of the grd cell t belogs to. The recostructo results (tragle meshes) are the put together the fal model. The recostructo s based o a oparametrc asotropc kerel-based method we developed for 3D surface recostructo from osy LIDAR data. I partcular, we proposed a asotropc kerel based oparametrc desty estmato method for outler removal, ad a asotropc desty ad ormal propagato techques for salecy feld extracto, ad a o-maxmum suppresso for 3D surface recostructo, whch we wll descrbe the followg... Asotropc kerel desty estmato Pots outsde the object surface are outlers that have to be removed. Sce the real object surface s ukow, t s hard to specfy a geeral crtero to detect outlers. We propose to employ parze-wdow based oparametrc desty estmato method for outler removal. Parze wdow based kerel desty estmato s the most popular oparametrc desty estmato method [4]. Gve data pots x, =,, the d-dmesoal Eucldea space

4 R d, the multvarate kerel desty estmate obtaed wth kerel K(x) ad wdow radus h, computed the pot x s defed as: f x) h d x x K( ) h ( () Wthout loss of geeralty, let s assume h = from ow o, so we could smplfy Equato () as: f ( x) K( x x ) () K(x) s geerally a sphercally symmetrc kerel fucto satsfyg: K( x) Ck, dk( x ), (3) where k(x) s the profle fucto of the kerel K(x). C k,d s the ormalzg costat such that K ( x), (4) ad K( x) dx d R, (5) x s the L orm (.e. Eucldea dstace metrc) of the d-dmesoal vector x. Employg the profle fucto otato, Equato () ca be further rewrtte as Ck, f ( x) d k( x x (6) There are three types of commoly used sphercal kerel fuctos K(x): the Epaechkov kerel, the uform kerel, ad the Gaussa kerel. The Epaechkov kerel s defed by the profle fucto k E (x): x ( x) The uform kerel s defed by the profle fucto k U (x): x x ) k E (7) ( x) x x k E (8) The Gaussa kerel s defed by the profle fucto k N (x): ( x) exp( x) x k N (9).. Outler removal by asotropc ellpsodal kerel For 3D pot cloud obtaed by laser sca, the outlers ted to spread the space radomly, whle real (we use a quotato here to emphasze the fact that the real surface s ukow) surface pots wll spread alog a th shell whch ecloses the real surface object. I other words, the dstrbuto of the outlers s relatvely sotropc, whle the dstrbuto of the real surface pots s rather asotropc. Hece we propose to employ a asotropc ellpsodal kerel based desty estmato method for outler removal. For asotropc kerel, the L orm x-x Equato (6), whch measures the Eucldea dstace metrc betwee two pots x ad x wll be replaced by the Mahalaobs dstace metrc x-x M : x x here H s the covarace matrx defed as: ad M t (( x x ) H ( x x )) /, () H DD T, () D = (x - x, x - x,, x - x). ()

5 t Geometrcally, ( x x ) H ( x x ) s a three-dmesoal ellpsod cetered at x, wth ts shape ad oretato defed by H. Usg Sgle Value Decomposto (SVD), the covarace matrx H ca be further decomposed as: H UAU T, (3) Wth 3 3 A (4) are the three egevalues of the matrx H, ad U s a orthoormal matrx whose colums are the egevectors of matrx H. To compute the asotropc kerel based desty, we apply a ellpsodal kerel E of equal sze ad shape o all the data pots. The oretato of the ellpsodal kerel E wll be determed locally. More specfcally, gve a pot x, we calculate ts covarace matrx H by pots located ts local sphercal eghborhood of a fxed radus. Wthout loss of geeralty, we assume the radus s (whch ca be doe by ormalzg the data by the radus). The U matrx of Equato (3) calculated by the covarace aalyss s kept uchaged to mata the oretato of the ellpsod. The sze ad shape of the ellpsod wll be modfed to be the same as the ellpsodal kerel E by modfyg the dagoal matrx A as: A r, (5) r s half of the legth of the mmum axs of the ellpsodal kerel E. After the desty value s estmated, we remove all the pots whose estmated desty value s smaller tha a user-defed threshold...3 Oretato propagato by mmum spag tree The ormal estmated by the PCA algorthm mght ot be cosstetly oreted as there s a ambguty of 8 degree of the obtaed egevector. Thus we eed to coduct a oretato propagato algorthm based o mmum spag tree. Ths s doe two steps. Frst, wth each volumetrc grd cell, we coduct clusterg based o the ormal formato, ad choose the oretato of the largest cluster as the oretato wth the volumetrc grd cell. The we compute the average ormal wth each grd cell ad buld a graph wth each grd cell (cotag pots) as a ode, ad the weghts of edges betwee the adjacet grd cell are defed as - *, where ad are the average ormal of the two adjacet grd cells. The mmum spag tree s mplemeted based o Kruskal s algorthm, whch s a algorthm graph theory that fds a mmum spag tree for a coected weghted graph. Ths meas t fds a subset of the edges that forms a tree that cludes every vertex, where the total weght of all the edges the tree s mmzed. If the graph s ot coected, the t fds a mmum spag forest (a mmum spag tree for each coected compoet). Kruskal's algorthm s a example of a greedy algorthm. It works as follows:. create a forest F (a set of trees), where each vertex the graph s a separate tree. create a set S cotag all the edges the graph 3. whle S s oempty a. remove a edge wth mmum weght from S b. f that edge coects two dfferet trees, the add t to the forest, combg two trees to a sgle tree c. otherwse dscard that edge. At the termato of the algorthm, the forest has oly oe compoet ad forms a mmum spag tree of the graph. Ths algorthm frst appeared 956 the Proceedgs of the Amerca Mathematcal Socety (page 48 5), wrtte by Joseph Kruskal [5].

6 ..4 Desty ad ormal propagato Oce we obta a cosstet oretato of the ormal, we propagate the ormal ad desty to the whole volumetrc grd. More specfcally, we frst embed the LDAR data by a volumetrc grd, ad coduct the aforemetoed Parzewdow based kerel desty estmato for each LDAR pot based o ts Mahalaobs dstace to ts eghborg LIDAR data (Equato ()). We the propagate the desty at dvdual pots to the volumetrc grds based o the same asotropc formula. I other words, we create a volumetrc desty feld, wth the desty value correspodg to the probablty of belogg to the so-surface of the object...5 No-maxmum suppresso based 3D surface recostructo The usged desty feld s the coverted to a sged dcator fucto by calculatg the er product of the ormal vector at the grd ode wth the gradet of the desty feld at the grd ode. Fally, we employ so-surface extracto algorthm such as Marchg Cubes to extract the so-surface, dcated by the zero-value of the sged dcator fucto..3 3-D vsualzato We developed a prototype software system based o OpeGL ad Vsual C++ that ca automatcally recostruct the 3D scee of the teror of a buldg or other structure from pot clouds acqured by the groud-based LIDAR scaer. We developed a user-fredly Graphcal User Iterface (GUI) that allows the users to teractvely vsualze, avgate ad walk through the room from dfferet vew agles, zoom ad out, etc. The recostructed 3D scee ca be exported the OBJ data format that s fully compatble ad exportable to other commercal vsualzato software. Moreover, we also developed a addtoal fuctoalty the prototype software system: automatcally geeratg the floor pla of the buldg. Ths addtoal fuctoalty ca be very useful for the user to have a hgh level uderstadg of the teror of the buldg ad s complemetary to the teractve 3D vsualzato provded by our software system.. CONCLUSION We developed a prototype system that ca automatcally recostruct 3D scees of the teror of a buldg, cave or other structure usg groud-based LIDAR scag techology. We develop a user-fredly real-tme vsualzato software package that wll allow the users to teractvely vsualze, avgate ad walk through the room from dfferet vew agles, zoom ad out, etc. REFERENCES [] M. A. Fschler ad R. C. Bolles, Radom Sample Cosesus: A Paradgm for Model Fttg wth Applcatos to Image Aalyss ad Automated Cartography. Comm. of the ACM 4: , 98. [] D. Comacu ad P. Meer, Mea Shft: A Robust Approach Toward Feature Space Aalyss. IEEE Trasactos o Patter Aalyss ad Mache Learg Vol. 4, No. 5, pp , May. [3] Ye Dua, Lu Yag, Hog Q, ad Dmtrs Samaras, Shape Recostructo from 3D ad D Data Usg PDE-Based Deformable Surfaces. Proceedgs of The 8th Europea Coferece o Computer Vso (Computer Vso - ECCV 4), Part III, May -4, 4, Prague, Czech Republc, pages (Lecture Notes Computer Scece 33). [4] Rchard O. Duda, Peter E. Hart, ad Davd G. Stork, Patter Classfcato, Wley-Iterscece, d edto, October. [5] Joseph. B. Kruskal, O the Shortest Spag Subtree of a Graph ad the Travelg Salesma Problem. Proceedgs of the Amerca Mathematcal Socety, Vol 7, No., pp. 48 5, Feburary 956.

7 (a) (b) (c) (d) Fgure : Herarchcal LIDAR data segmetato. (a) Orgal LIDAR data; (b) Floor (blue color), celg (gree color), ad vertcal walls (red, cya ad mageta colors) detfed; (c) Idvdual objects extracted (show dfferet colors); (d) Each object s segmeted to homogeous clusters (show dfferet colors).