STUDY ON 3D TEXTURED BUILDING MODEL BASED ON ADS40 IMAGE AND 3D MODEL

Transcription

1 STUDY ON 3D TEXTURED BUILDING MODEL BASED ON ADS4 IMAGE AND 3D MODEL Liu Zhen,, *, Gong Peng c, Shi Peijun Ssgw T d College of Resource Science & Technology, Beijing Norml University,1875, liuzhen@nu.edu.cn Center of Informtion & Network Technology, Beijing Norml University,1875 c Deprtment of Environmentl Science, Policy, nd Mngement, Cliforni University t Berkeley, USA d Institute of GIS, Psco Corp, Tokyo KEY WORDS: 3D Textured model, ADS4 imge, Feture extrction, Feture mtching ABSTRACT: An utomtic method of 3D textured uilding modelling with n irorne liner scnner sensor imge nd 3D model is proposed, which includes to extrct uildings feture from imge, to mtch the extrcted feture with 3D mode nd to mp texture ccording to the correspondence etween imge nd model. Feture extrction from imge nd feture mtching with 3D model re two key steps to uild 3D textured uilding model utomticlly. In the rticle, new line extrction pproch through multi-level feture filter is presented, which is consisted of Cnny edge detector, edge phse filter, edge direction filter with fult tolernce, Hough trnsformtion, neighour line segment fusion. Bsed line segmenttion, prllelogrm extrction lgorithm pplying perceptul orgniztion is proposed, which is sed on perceptul orgniztion, employing uncertinty resoning nd shpe expression form sed on prt to otin simplicity, low computtionl overhed, nd noise free. Mtching etween 2D imge nd 3D model is in essence to find trnsformtion mtrix to minimize the error. A lot of lgorithms re presented to solve the mtch prolem with severl unknown prmeters, ut still no good solution to prolems with too much unknown prmeters. Projected imge will first e outputted sed on cmer model fter prtil mtching etween prllelogrm extrcted nd 3D uilding. Then Husdorff distnce i clculted etween edge imge nd projected imge. Bsed on which, feture mtching is finished to extrct texture nd to mp texture to uildings. 1. INTRODUCTION Virtul Relity sed on 3D uilding re successfully employed tody for rel-time visuliztion in such diverse fields s urn plnning, city plnning nd rchitecturl design. Visulistion of 3D uilding is generl tsk of nerly ll science nd engineering disciplines to esily ssess complex oject fetures, oject grouping, environmentl goodness-of-fits nd mny other pplictions. Texture mpping for 3D uilding ecomes n issue when 3D uilding reconstruction is desired. It is out of question tht 3D textured uildings cn mnully nd semiutomticlly reconstructed using photogrmmetric stereo restitution processes. Nowdys, mny softwre pckges offer methods for rectifiction nd texture mpping, ut very often the hndling is time-consuming nd circumstntil [1]. There re few exmple of utomtion 3D textured uilding modelling. Previous reserch especilly from University of South Cliforni nd University of Stuttgrt, hs found tht ecuse of noise, occlusion nd lck of cmer informtion, extrcting uildings from urn re monoculr eril imges is not fesile [2]. While mtching 3D models with 2D imges is lso difficult without knowing nything out the cmer prmeters. An utomtic method of 3D textured uilding modelling with n irorne liner scnner sensor imge nd 3D model is proposed, which includes extrcting uildings fetures from imge, to mtch extrcted fetures with 3D mode nd to mp texture ccording to the correspondence etween imge nd model. Feture extrction from imge nd feture mtching with 3D model re two key steps to uild 3D textured uilding model utomticlly. In the pper, prllelogrm extrction lgorithm sed on the theory of perceptul orgniztion, employing uncertinty resoning nd shpe expression form sed on prt to otin simplicity, low computtionl overhed, nd noise free. The methods re illustrted with n irorne liner scnner sensor imge over the downtown of Shinjuku in Tokyo city, Jpn. The result of uilding texture mpping explins the method of uilding outline extrction nd mtch etween 2D feture nd 3D model very well DATA USED The multi-spectrl dt used were cquired y ADS4 SP1 with the resolution of.2m, which hs 4 nds, nmely NIR, R, G nd B chnnel covered the downtown of Shinjuku of Tokyo. The other dt used is 3D model CAD file (DXF) of this region. Edge extrction Edge Detector 3. APPROACH The Cnny edge detector hs een shown to e optiml for imges corrupted y Gussin white noise [3], nd is used to * Corresponding uthor. 1

2 detect edges in the current work. It is demonstrly more effective thn the LoG opertor, Soel opertor nd others. Using Cnny Edge detector, we otin not only edge intensity informtion, ut lso edge phse, which is used in the ltter step. Phse filter It resonly ssume tht neighour pixels in stright contour of mn-mde oject would hve similr phse. Thus we designed the lgorithm s follows: Lel edge pixels with phse: For ech pixel, lel surrounding pixels, 1, 2 7, nd ssign ech its phse. We needn t clculte n inverse trigonometric function for ech edge pixel, ut only pply ddition nd multipliction on (derivtive in X xis), (derivtive in Y xis) nd invrint tn(pi/8), to reduce the computtionl overhed. Trck edge: For ech pixel check its 8 neighouring pixels. If neighour pixel Pn hs phse similr to then we go on trcking long Pn. If this trcking results in too few pixels decide y threshold vlue, we cn resonly judge it to e noise edge nd delete it. Edge filter ccording direction The ove filter process hs no effect on curve edges whose pixels phses don t chnge ruptly. Therefore, further filtering is necessry. For line-supported edge, connections etween neighour pixels re typiclly in the sme direction. Figure 1 shows tht neighour pixels only hve 8 connection directions t most nd reciprocl oscilltion phenomenon would pper. 3.2 Line segment extrction Line segment extrction will e finished y line comintion fter Hough trnsformtion. For ech filtered line-support edge discovered y the ovementioned trcking process, Hough Trnsformtion (HT) is pplied, which results line segment. HT is stle nd roust line detection method [8], ut its primry disdvntge is its lrge computtionl overhed. In the pper HT is modified to reduce overhed: I) Clculte the centre of ech line-supported trck result. Set new frme with the centre s its origin point nd express the trck result in this new frme. Thus, the possile rdius rnge is reduced to locte in smll window round the centre. II) Estimte the possile ngle rnge ccording the retined direction code. III) Apply HT, select the prmeter with mx vote, then select the pixels who vote for it. Then we determine the two end points of the extrcted line ccording the mx nd min coordintes mong the selected pixels. IV) To express the line in the originl frme. This scheme voids severl disdvntges of HT, such s: rpid performnce reduction with lrger imge size, difficulty determining the vote threshold when there re multiple lines to e extrcted, nd difficulty determining the end points of n extrcted line. The generlity of HT, however, is lost in this process, so further processing is necessry. For ech extrcted stright line, serch the neighour re of its two end points. c Figure 1-2 Direction oscillte. As line is trcked, direction fult tolernce is pplied s Tle Tle 1 Line trck direction I) For n edge pixel trcking in given direction. For ech neighour pixel locted on the D direction of if direction D hs not een trcked, then go on trcking, set direction fult counter C to, nd dd the temporry trck result to the finl trck result. Otherwise trck edge pixels locted in neighour direction of D, increse direction fult counter C, nd put this trck result in temporry uffer. Retin the direction code. II) If counter C is greter thn threshold, then ndon the trck result in uffer. III) If the finl trck result hs too few pixels, then the edge is considered to e noise, which is not line-supported. This method is similr to some chin code sed line detection technologies [4-7], ut it is more efficient nd hs more powerful cpilities in reducing noise.. Figure 2 Line comintion If nother line is found, nd the two lines proceed in similr direction, then comine the two lines. For ech line in line set, if nother line is founded tht cn e comined with it in the sme set; the line set cn e comined. Through this process, short lines, such s line nd in Figure 2, re comined into long lines, nd mny noise lines (such s line c in Figure 2) re removed. 3.3 Prllelogrm extrction of uilding To extrct prllelogrms the theory of perception orgniztion nd uncertinty resoning re employed. Perception orgniztion, first proposed y Lowe [9] in his SCERPO system, hs een recognized y mny reserchers for its ility to derive glol structure from locl primitives. It is pplied in numerous projects due to its low computtionl overhed nd high nti-noise cpilities [1-12]. Bsed on the chrcteristics of prllelogrms, the following lgorithm, shown in Figure 3, is used to dpt perception orgniztion to extrct prllelogrm. Uncertinty resoning is lso pplied to solve the following prolem. Although Byesin resoning nd Dmpster-Shfer s evidence theory re used y others [13-15], simpler ut effective method is used in the pper. The comintion of multi evidence is descried elow: 1) Bel( A1 nd A2 nd An )=min{ Bel(A1),Bel(A2), Bel(An) }; 2

3 2) Bel( A1 or A2 or An )=mx{ Bel(A1),Bel(A2), Bel(An) }; Bel(X) denotes elief for X; Figure 4 Corner extrction Figure 3 Prllelogrm extrction Shpe representtion: All shpes, including corners, U-shpes nd prllelogrms re represented y prts defined s follows. Define: For point set P in 2D plne, the two end points of line segment is point1 nd point2, nd the coordintes of n ritrry point in line is (x, y).if point1.x-point2.x < pont1.ypoint2.y,then point set {( p, y) p < x,( p, y) P} is clled the left side of line, {( p, y) p > x,( p, y) P} is clled the right side of line, nd two topple (, pos) is clled prt, where pos is flg to denote left or right. In similr wy, we cn define the cse point1.x-point2.x > point1.y-point2.y. Thus corners, U shpes nd prllelogrms cn e expressed y 2 prts, 3 prts nd 4 prts Corner extrction: For possile corner C1 shown in Figure 4, the elief cn e clculted s following. Bel 1 = 1 dist( P) / dist( P 1) = 1 dist( P 1) > lenth, 1 Bel 2 = else, Bel = dist( P ) / dist( P ) Bel dist( P ) > lenth, 1 2 = else, Here dist(p1,p2) denotes the distnce etween point p1 nd p2,bel(x) is the elief of X nd length is the minimum side length of n ccepted corner. Then the elief of C1 cn e expressed s: Bel ( C1) = min{ Bel, Bel 1, Bel, Bel 1}. If elief threshold is given, then ny corner with elief greter thn the threshold is ccepted U shpe nd prllelogrm extrction: As Figure 5, corner structure C1 consisting of prt1=(, pos_),prt2=(, pos_), if nother corner structure C2 consisting of prt3=(c, pos_c) nd prt2 exists, then we cn compute the elief of the prllel structure P consisting of prt3 nd prt1 s follows: If is not locted in pos_c side of c, or c is not locted in pos_ side of then Bel(p)=; Otherwise, 1 ngle(, c) / Angle, ngle(, c) < Angle Bel( P) = {, else, where ngle(,c) denotes the inclintion of nd c nd Angle is the mx inclintion etween two lines in n ccepted prllelogrm structure. Figure 5 U shpe nd prllelogrm extrction Prllelogrm complementtion: For n extrcted U-shped U, if no prllelogrm is extrcted from it nd Bel(U) is greter thn given threshold, it could ssume tht in fct it is contour of prllelogrm, ut the forth side hs een lost ecuse of line extrction. The prllelogrm cn e closed with fourth side d such s Figure 6. c d Figure 6 Prllelogrm complementtion Solve conflict: For those extrcted prllelogrms, spce conflicts could pper. The following rules re employed to solve this prolem. i) Block off rule: s shown in Figure 7(), oth, nd.c cn form corner structure, ut ecuse locks nd c, the corner formed y nd c should e deleted. ii) Outer contour rule: s shown in Figure 7 (), prllelogrm (P1,P2,P5) is contined in prllelogrm (P1,P3,P4), so the former is deleted. 3

4 iii) Neighour sor rule: s shown in Figure 7 (c), side is line generted s prllelogrm complement, nd it is ner to nd hs similr direction s. Therefore, should e sored y ; tht is, we should delete nd enlrge. iv) Higher elief rule: s shown in Figure 7 (d), the prllelogrm with lower elief should e deleted. P4 P3 model-projecting imge, for ech position ( u,v o ), clculte the Husdorff distnce, where the position which hs the minimum Husdorff distnce is the optiml mtching result. Husdorff distnce cn e defined etween point sets, line segment sets nd prllelogrm sets with lower computtionl overhed, yet the reliility of the process lso drops. The following flowchrt, shown in Figure 8, is employed to finish mtch etween feture imge nd 3D model. P5 P2 c P P1 () lock off rule () outer contour rule (c) neighour sor rule (d) higher elief rule Figure 7 Conflict rules 3.4 Mtch etween 2D imge nd 3D model Mtching etween 2D imges nd 3D models is in essence the serch for trnsformtion mtrix tht minimizes error, which is lso very importnt nd difficult tsk to chieve texture mpping for uilding. In the pper, cursory mtch will e done sed on cmer prmeters efore fine mtching. Cursory mtch cn limit mtch in smll rnge to reduce clcultion s well s error of mtching. The trnsformtion mtrix of perspective projection cn e written explicitly s function of its five intrinsic prmeters ( α, β, u, nd θ ) nd its six extrinsic prmeters (the tree ngles defining rottion mtrix R nd the three coordintes defining trnsltion vector t ),nmely, r 1 r 2 T T T where,,nd r 3 denote the three rows of the mtrix R nd t x,t y,nd t z re the coordintes of the vector t [2]. There re 11 prmeters to solve for. If informtion out these prmeters is not ville, the prmeter spce is very lrge nd computtionl overhed would e uncceptly lrge. Vrious reserchers hve proposed lgorithms to solve the mtch prolem with severl unknown prmeters, ut no solution exists to solve prolems with mny unknown prmeters [17-19]. So the lgorithm will simplify the cmerl model ccording to the imge nd CAD dt given. E.g., α = β, θ =, R = I (I is unit mtrix), setting u,v to the centre of the eril imge nd so on. Thus prmeters unknown re fewer, nd the difficulty will come down, so to mke the lgorithm prcticl. Bsed on cmer model prmeters, for n ritrry prt of the eril imge, only u nd v re unknown. It cn move the eril imge long the 3.5 Texture Mpping Figure 8 Mtch flowchrt Since the cmer positions of the originl photogrphs re recovered during the modelling phse, projecting the imges onto the model is stright forwrd. The process of texturempping single imge onto the model cn e thought of s replcing ech cmer with slide projector tht projects the originl imge onto the model. When the model is not convex, it is possile tht some prts of the model will shdow others with respect to the cmer. While such shdowed regions could e determined using n oject-spce visile surfce lgorithm, or n imge-spce ry csting lgorithm, which is efficiently implemented using z-uffer hrdwre. 4. RESULT The ove lgorithms re employed to finish 3D texture mpping for uildings in Shinjuku of Tokyo, Jpn. The result is shown in Figure 9. From Figure 9(), there re so mny flse edges fter edge filtering nd Hough trnsformtion. But fter line comintion, most of residul edges re outlines of uildings. From Figure 9(d), some prllelogrms of uildings un-shdowed sides re extrcted, while some of them re not resumed. Bsed some of prllelogrms, prtil mtch cn e done. After mtching etween 2D imge nd 3D model, 3D textured uildings cn e chieved, which is shown in Figure 1. The experiment results suggest the lgorithms of 3D textured uilding modelling sed on mtching etween 2D feture imge nd 3D model is relile nd efficient. But there re still some errors in the result of texture mpping such s incorrect of texture mpping position. By nlysis of result, errors re found 4

5 to e derived from the following fctors: 1) error from CAD model given; 2) errors cused y ADS4 sensor in flying. 3) errors from clcultion. extrction nd mtching to e more roust to noise y using rtificil intelligence. () Test Buildings imge () Result of Line Comintion REFERENCE A Gruen, Zhng L, Wng X. 3D City Modelling with TLS (Three-Line Scnner) Dt. Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Vol. XXXIV-5/W1 F Dieter. 3D Building Visulistion Outdoor nd Indoor Applictions, in Fritsch (Ed.). Photogrmmetric Week '3. Wichmnn Verlg, Heidelerg, pp J F Cnny. A computtionl pproch to edge detection. IEEE Trns. Pttern Anlysis nd Mchine Intelligence, 1986, 8 (6) : Zhu Xio-yue. Extrct Stright Line From Imge Bsed on Computer Vision Tsk. Journl of Nnchng Institute of Aeronuticl Technology (Nturl Science).Sep. 23.Vol.17,No.3:63-66 Bo su-su.yng Lu. Algorithm for Detecting Verticl nd Horizontl Lines in Rel-time Imge processing. Journl of Hefei University of Technology. Aug 23. Vol.26, No.4: Sun Ai-rong, Tn Yi-hu. Journl of Wuhn University of Technology (Trnsporttion Science & Engineering ).Dec.23. Vol 27, No.6: (c) Result of Corner Extrction (d) Result of Prllelogrm Extrction Figure 9 Building Fetures Extrction Shi Ce, Xu Sheng-rong, Jing Ren-jie etc. The Replce Algorithm of Hough Trnsform in Rel-time Imge Processing. Journl of Zhejing University (Engineering Science). Sep,1999. Vol 33, No.5: Luc Bron, P. Eng. Genetic Algorithm for Line Extrction. Rpport technique EPM/RT-98/6, École Polytechnique de Montrél, 2 pges, oût Dvid G. Lowe, Perceptul Orgniztion nd Visul Recognition, Kluwer Acdemic Pulishers, Norwell, MA, 1985 Chung-An Lin. Perception of 3-D Ojects From n Intensity Imge Using Simple Geometric Models. PhD thesis,fculty of the Grdute School, University of Southern Cliforni.1996:28. Figure 1 Result of 3D Textured Building 5. CONCLUSION The lgorithm of prllelogrm extrction sed on perceptul orgniztion theory fits to resolve the prolem of feture extrcting for uildings using high sptil resolution imgery with more detils nd noises. Reltion etween coordintes of imge nd coordintes of 3D model cn e uilt fter mtching etween 2D imge nd 3D model. The experience suggests tht utomtion 3D textured uilding modelling proposed in the pper is prcticl. Especilly given cmer model, mtching result will e more exct. There re lso mny studies to continue, such s how to improve the lgorithm of feture L.R.Willims,A.R.Hnson.Perceptul completion of occluded surfces.computer Vision nd Imge Understnding,64(1):1-2,July Christopher O.Jynes,Frnk Stolle,Roert T.Collins.Tsk Driven Perceptul Orgniztion for Extrction of Rooftop Polygons. Applictions of Computer Vision, 1994., Proceedings of the Second IEEE Workshop on P.Vsseur,C.Pegrd. Perceptul Orgniztion Approch Bsed on Dempster-Shfer Theory. Pttern Recognition,Volume 32, Issue 8, August 1999: Mei Xue-ling. Automtic 3D Modeling of Regulr Houses from Aeril Imgery. Dr. thesis. Wuhn Technicl University of Surveying nd Mpping

6 Xi Xue-qing. Reserch on Model Bsed 3-D Oject recognition in remote sensing imge. Dr. thesis. Ntionl University of Defense Technology,2. Dvid A. Forsyth, Jen Poince. Computer vision: A Modern Approch. Tsinghu University Press, Beijing, 24: Dvid.Jcos, Ronen Bsri. 3-D to 2-D Recognition with Regions Conference on Computer Vision nd Pttern Recognition (CVPR '97) Ronen Bsri, Dphn Weinshll. Distnce Meteric etween 3D Models nd 2D Imges for Recognition nd Clssifiction. IEEE Trns. On Pttern Anlysis nd Mchine Intelligence, 18(4):465-47,1996. Steven Gold, Annd Rngrjn, Chien-Ping Lu, etc. New Algorithms for 2D nd 3D Mtching: Pose Estimtion nd Correspondence. Pttern Recognition 31(8): (1998) 6