Pocket3D Designing a 3D Scanner by means of a PDA 3D DIGITIZATION

Transcription

1 Pocke3D Desgnng a 3D Scanner by means of a PDA 3D DIGITIZATION Subjec: 3D Dgzaon Insrucor: Dr. Davd Fof Suden: AULINAS Josep GARCIA Frederc GIANCARDO Luca Posgraduae n: VIBOT MSc

2 Table of conens 1. Inroducon Wha s a 3D scanner? D Scanner s classfcaon Theorecal background D merology The prncple of rangulaon Laser peak deecon New mehod for obanng pon correspondences Pons on he upper and lower edges Pons on he laser srpe Calbraon mehod usng a complee quadrangle Camera localzaon (self-referencng) Implemenaon Acquson of mages, he nal seup Calbraon of he camera wh he complee quadrangle n Malab Projec under LabVIEW Projec under Mcrosof Vsual Sudo (C#) Image acquson from he camera Laser deecon algorhms, mplemenaon and esng Frs algorhm Second algorhm Camera pose esmaon Tag Deecon algorhm Camera homography esmaon Fnal Implemenaon Conclusons

3 Absrac The objecve of hs projec s o develop a laser-based rangulaon 3D scanner on a PDA (Personal Dgal Asssan). The sysem, conssng of a PDA wh a camera and a laser beam, mus be auonomous. Calbraon of he camera, laser lne deecon and rangulaon has been mplemened as a sand-alone applcaon. 1. Inroducon 3D nspecon, 3D reconsrucon and n general 3D magng are ho opcs whch more and more are appearng n he ndusry. There s a clear need for buldng 3D models of he vsble surface for a wde varey of objecs n shape, sze and ype of maeral. Applcaons requre nspecng, documenng, reproducng, or smply provdng he capably o remoely observe he represenaon of a real objec under any seleced vewpon. Owng o hese facs, 3D scannng s currenly n a fas developmen sage. Ths projec deals o a hand-held range sensor, whch s a useful ool n many suaons. Snce s manoeuvrable, s easy o selec he area o be measured whou he consran on moon mposed by a ranslaon or roaon sysem. In addon, s easy o oren he sensor relave o he surface o ge he opmal coverage (samplng) or he opmal condons for measuremen qualy. From desgn o mplemenaon, several seps mus be performed n order o acheve he fnal goal, a 3D scanner. These seps nvolve a wde range of heory, such as deecon of feaures, eppolar geomery, rangulaon or reconsrucon from 2D mages. Theory whch n par has been suded whn he maser lecures. Manly for he possbly o apply and mprove our knowledge over hs feld and also because s an ambous approach we seleced hs projec. As far as he repor srucure s concerned, chaper one sars by nroducng he concep of a 3D scanner, s funconales and s classfcaon. Second chaper presens he heorecal background, how he sysem calbraon s performed, how o deec a laser srpe n he mage, he rangulaon prncples n order o compue 3D coordnaes from 2D projecons, and how o locae he camera durng he scannng process (o generae a sngle cloud of 3D pons by regserng he paral clouds of pons). The modellng, smulaon and mplemenaon of he sysem s descrbed n secon hree Wha s a 3D scanner? A 3D scanner s a devce ha analyzes a real-world objec or envronmen o collec daa on s shape and possbly s appearance (.e. colour or exure). The colleced daa can hen be used o consruc hree dmensonal models useful for a wde varey of applcaons. These devces are used exensvely by he eneranmen ndusry n he producon of moves and vdeo games. Oher common applcaons of hs echnology nclude ndusral desgn, reverse engneerng and prooypng, compuer vson and documenaon of culural arefacs. Many dfferen echnologes can be used o buld hese 3D scannng devces; each echnology has s own lmaons, advanages and coss. The range of objecs ha can be dgzed s sll lmed: for example opcal echnologes encouner many dffcules wh shny, mrrorng or ransparen objecs

4 1.2. 3D Scanner s classfcaon There are manly hree echnologes n whch 3D scanners can be classfed: conac 3D scanners, non-conac acve 3D scanners and non-conac passve 3D scanners. Brefly, a conac scanner (Fgure 1.1) probes he subjec hrough physcal ouchng as for example a CMM (Coordnae Measurng Machne). Ths knd of scanners presens several dsadvanages such as he possbly of damagng or modfyng delcae or valuable objecs durng he scannng process or he slow processng o oban he 3D model. In fac, he movng arm has o ouch all he area of he objec o be scanned, hence akes sgnfcanly more me compared o oher scannng mehods. Ths projec s enclosed n he second group of 3D scanners: non-conac acve scanners (Fgure 1.2). Acve scanners em some knd of radaon or lgh and deec s reflecon n order o probe an objec or envronmen. Possble ypes of emssons use lgh sources, ulrasound or X-ray. Fnally, he las group s non-conac passve scanners (Fgure 1.3) whch do no em any knd of radaon by hemselves, bu nsead rely on deecng refleced amben radaon. Mos scanners of hs ype deec vsble lgh because s a readly avalable amben radaon. Oher ypes of radaon, such as nfrared could also be used. Fgure D conac scanner from Faro enerprse whch can scan objecs unl 2.4m wh a resoluon of +/-18µm. Fgure D non-conac acve scanner from Mnola enerprse. The sandard verson can scan objecs from.6 o 1.m wh a resoluon of.5mm. Fgure D non-conac passve scanner from RSI GmbH whch apples sereovson o oban he 3D pons. 2. Theorecal background 3D laser scanners are oday's mos accurae non-conac acve measurng ools. However, here are wo mporan ssues o be consdered. The former concerns he denfcaon of he geomercal parameers of he sysem, snce hey allow he equaons o be denfed and solved n order o provde he reconsruced 3D pons. The process o oban hese values s known as calbraon, and s descrbed n Secon 2.5. The second problem concerns he laser srpe deecon. I s worh regardng ha he geomerc fgures ha are reconsruced are pons, and pons are zero-dmensonal, ha s, hey do no have volume or surface. I s evden ha he sze of a laser spo s, by no means, an nfnely small pon. Insead, s an elecromagnec radaon ha can be focused down o abou 2µm usng he curren lens manufacurng echnology. Hence, seems very sensve o provde a means of deecng a sngle pon as much represenave of he laser spo as possble. The lgh energy dsrbuon of he srpe follows a Gaussan - 4 -

5 paern, hence s assumed ha a good canddae for hs represenave pon s he lgh peak of he Gaussan paern. For hs reason, he echnques or algorhms for obanng he pxel of he mage ha bes quanfy hs pon are called numercal peak deecors. The need for a projecve geomery-based calbraon procedure s clear, and several auhors have conrbuons o hs end. Wh hs approach, s no necessary o use accurae mechancal posonng pars for denfyng he laser plane/lne equaon, and hs feaure s very neresng when he calbraon n hazardous envronmens, such as underwaer applcaons, s needed D merology Merology s he scence of measuremen, embracng boh expermen and heorecal deermnaons a any level of uncerany n any feld of Scence and Technology. In parcular, 3D merology s a very neresng echnology and s beng appled n dverse felds such as ndusry, 3D medcal magng, human body measuremen or robocs. One of he approaches o 3D sensng by non-conac devces s he wdely spread acve 3D scanner devces. They are called acve, because hey change he lghng of he scene by projecng srucured lgh on. The way hey oban 3D nformaon s by usng a camera for measurng he dspary beween he projeced srucured lgh and he mage of he scene. Among he srucured lgh devces, hs projec s focused o he case of rangulaon based 3D laser scanners. Fgure 2.1 shows a op vew of he basc dagram whch bulds up such sysem. Noe ha here appear mporan conceps and quanes lke base lne, opcal axs, laser axs, angles and wh respec o he vercal, quanfyng he orenaon of he eye and laser, ec. These quanes model he geomerc aspecs of he sysem. Fgure 2.1. Basc 3D laser scanner. The eye appearng n ha square s obvously replaced by a camera n real applcaons. Noe ha, jus lke n he case of he human vson sysem, he poson and orenaon of he eye/camera s mporan for shape/deph sensng, as well as he orenaon and poson of he laser projecon. In addon, a laser srpe or spo can be narrowly focused even a grea dsances, hence provdng even more dreconaly han he fovea vson

6 2.2. The prncple of rangulaon Ths secon nroduces he prncple of rangulaon focusng s use n he laser sl scannng. Generally hs prncple s accompaned by he called correspondence problem, whch s sll unsolved n many cases. The ypcal suaon n whch such prncple s appled s shown n fgure 2.2 where a pon P n 3D space s seen by wo dfferen cameras. If he spaal relaonshp beween boh cameras s accuraely known, he poson of P can be found by s wo projecons ( p ' and p '') on he correspondng mage planes, compung he nersecon of lnes P p' and P p' '. Fgure 2.2. Trangulaon prncple usng wo cameras. The spaal relaonshp beween boh cameras can be easly found by applyng a sandard camera calbraon procedure o be chosen among he mulple exsng ones, dependng on how dsored are he capured mages. However, he bgges problem s how can P be unquely denfed n boh mage planes?.e. how sure can be saed ha boh p ' and p '' correspond o he projecon of P? Ths problem s wdely known as he correspondence problem, whch s very dffcul o solve robusly n mulple camera sysems, unless hghly defned corners are maged. Besdes, one of he echnques ha works very well for denfyng pons n 3D space s he use of Acve Vson. The erm acve means ha some well known lgh paern (ypcally laser or coded/uncoded srucured lgh) n dfferen geomeres (pons, planes, crosses, parallel bars, ec.) s projeced ono he scene, so an llumnaed pon n he 3D space s well seen by boh cameras, and hence, he 3D coordnaes of P are easly compued by rangulaon. In addon, f he lgh source geomery, ha s, s pose and scannng movemen, s well known, one of he cameras can be replaced by he lgh source and 3D nformaon can also be obaned by rangulaon. Ths procedure s wdely used by commercal and research prooype range fnders. Fgure 2.3 shows how camera 2 n fgure 2.2 has been replaced by a sweepng laser lgh source. Usng hs echnque he camera model has o be compued usng a calbraon procedure, and he lne or plane laser equaon mus be known n order o compue s nersecon wh he lne P p'

7 Fgure 2.3. Laser scanner sysem wh scannng lgh source Laser peak deecon The accuracy of a 3D reconsrucon usng laser scanners s sgnfcanly consraned by he deecon of he laser srpe n he mage. Snce he energy paern of such a srpe corresponds o a Gaussan profle, makes sense o deec he pon of maxmum lgh nensy (or peak) by compung he zero-crossng pon of he frs dervave of such Gaussan profle. However, because nose s presen n every physcal process, such as elecronc mage formaon, s no sensve o perform he dervave of he mage of he srpe n almos any suaon, unless a prevous flerng sage s done. Consderng ha srpe scannng s an nherenly row-parallel process, every row of a gven mage mus be processed ndependenly n order o compue s correspondng peak poson n he row. The opcal properes of he surface sgnfcanly deermne he performance of he laser scanner. The opmal surface ype for scannng purposes s a oally lamberan surface wh a hgh reflecon ndex. Translucd surfaces are ofen presen n our everyday lfe (ceran ypes of plasc, anmal ssue, slcon, resns, ceran rocks or mnerals, ec.). Fgure 2.5 shows how a ray of lgh behaves when mpnges such knd of surface. In a ranslucd surface, lgh reflecs as n a lamberan surface, bu goes hrough he maeral unl a ceran deph. The hgher he lgh power, he deeper he lgh peneraes nsde he maeral. In addon, he lgh scaers nsde he maeral, so ha a camera lookng a "sees" laser reflecons sourcng from nsde. Fgure 2.4 (rgh) shows a laser srpe refleced on a lamberan surface, whle fgure 2.4 (lef) shows how he reflecon on a ranslucd surface s seen by he camera. As s shown, a laser srpe mpngng on a ranslucd surface nduces a lo of undesred lgh peaks where hey are no expeced o be. In addon, f he lgh power s lowered, he nose due o he dfferen sources becomes more and more sgnfcan and hence, he reconsrucon qualy degrades

8 (a) (b) Fgure 2.4. Behavour of lgh refleced on a specular surface (a), and on a lamberan surface (b). Fgure 2.5. Behavour of lgh refleced on a ranslucd surface. Three nose sources have been found o nfluence he hree-dmensonal measuremen of camera-laser based 3D scanners: elecrcal nose, quansaon nose and speckle nose. The speckle s drecly relaed o he naure of laser lgh, whle he oher wo nose sources are nheren o he mage sensor. Speckle s due o he reduced wavelengh of lgh compared o he surface roughness and he monochromacy of laser lgh and nfluences he 3D measuremen. These hree nose sources are combned ogeher and make he observer see he consrucve and desrucve nerferences whn he laser srpe. Fgure 2.6. Laser srpe of a ranslucd (lef) and a lamberan (rgh) surface

9 2.4. New mehod for obanng pon correspondences Any se of four non-algned pons A, B, C, D on he plane can be joned par wse by sx dsnc lnes. Ths fgure s called he Complee Quadrangle and exhbs several useful properes n compuer vson applcaons. Sx of he pons defned by hs fgure are used n order o generae 3D pons for calbraon and her correspondences on he mage plane, n oher words, from hs complee quadrangle, he ransformaon marx whch maps he laser srpe pons on he mage plane o he world coordnaes ( W T I ) s calculaed Pons on he upper and lower edges Fgure 2.7 shows how he four lnes defned by pon F and pons A, P A, B, and G respecvely, confgures a pencl of lnes. Hence s sraghforward o oban her cross-rao, provded ha pon P A s known. However, P A s an unknown world 3D pon, wh only he lne ha conans beng known, ha s, only pons A, B, and G are known. Neverheless, from he fgure can be seen ha pons A, P A, B and G are all known. Snce he geomerc relaon beween pons A o G and pons A o G s projecve, he cross-rao can be compued wh pons A o G and s value can be used for compung he coordnaes of pon P A. The same procedure can be adoped beween pons D o G and D o G. By repeang he process for dfferen posons and/or orenaons of he laser srpe, can be seen ha a se of correspondences P A, P A and P B, P B can be esablshed wh hgh accuracy. Fgure 2.7. The cross-rao and he complee quadrangle

10 Pons on he laser srpe The esmaon of he 4x3 ransformaon W T I, whch maps pons on he mage plane ( I P) o pons on he laser plane ( W P), s performed from pon o pon correspondences. In subsecon 2.4.1, a mehod for obanng he nersecon pons of he laser plane wh he upper and lower lnes of he calbraon arge (P A, P B ) was exposed. However, only wo pons defne he laser srpe for each deph, as shown n fgure 2.8. I s reasonable o hnk ha he more pons per srpe he beer he parameer esmaon of W T I wll be. Lookng back o fgure 2.7, s clear ha he cross-rao of pons F, A, D and an arbrary 2D pon P L beween A and D can be obaned. Snce pons F, A and D are known, he prevously compued cross-rao can be used for calculang he coordnaes of he 3D pon P L correspondng o P L. Ths process s depced n fgure 2.9, where he pons on he lne A D are called P L. I s obvous ha a pencl of lnes L T can be defned beween pon G and pons P L, and ha hese lnes defne a se of nersecon pons P R on he lne B C. Now, for each of he lnes of he pencl L T, he nersecon wh he laser srpe P S can be obaned, and hence, he cross-rao beween pon G, P R, P S and P L can be calculaed. Fnally, snce he 3D pons G, P R and P L are known, he value of he cross-rao can be used n order o oban he se of 3D pons P S layng on he laser srpe. Fgure 2.8. The laser plane defned by upper and lower pons

11 Fgure 2.9. Pon generaon Calbraon mehod usng a complee quadrangle Once he pon correspondences have been denfed for each srpe poson, he parameers of he 2D o 3D mappng mus be esmaed. Equaon 2.1 shows he relaonshp beween pons on he mage plane I [u; v; 1] T wh 3D pons W [sx; sy; sz; s] T on he laser plane. sx sy = sz s u v 1 W T I (Eq. 2.1) As sad n 2.4, W T I s he ransformaon marx whch maps pons on he mage plane ( I P) o pons on he laser plane ( W P). Bu how can be compued. Chen and Kak [Chen87] borrowed hs resul and used a homography for modellng he geomerc relaonshp beween he laser plane and he mage plane. However, a homography relaes 2D pons on a plane o 2D pons on a second plane and does no solve he process of mappng from 2D pons o 3D pons. Wh hs am, Chen demonsraed ha a 3D coordnae sysem could be added o he laser plane, such ha pons on he laser plane expand naurally o 3D coordnaes by addng a hrd componen equal o zero o he 2D coordnaes

12 {W} World (reference) coordnae sysem {I} Image coordnae sysem {L} Laser coordnae sysem {L 2 } Bdmensonal coordnae sysem Fgure 2.1. Geomerc scheme of Chen & Kak's mehod. Where {W} s he world or reference coordnae sysem, {I} s he mage coordnae sysem, wh uns expressed n pxels, and {L} s he laser coordnae sysem. {L 2 } s a b-dmensonal coordnae sysem, where he x and y concde wh he x and y axs of {L}. As shown n equaon 2.2, he geomery of he whole sysem can be modelled by a 4x3 ransformaon W T I wh 11 degrees of freedom. Ths ransformaon allows he pons on he laser plane o be mapped o 3D coordnaes, wh respec o {W}. x y W L = TL TL z w u v = 1 1 e e 1 e e u = v 1 u v = 1 u L W 23 W H 2 I TL 21 e22 e23 TI v e32 e e (Eq. 2.2) Obvously, he parameers 11 o 34 should be esmaed as precsely as possble, n order o maxmse he accuracy of he reconsrucon. Accordng o equaon 2.2, he expressons for sx, sy, sz and s are obaned and shown n equaon 2.3. sx = sy = sz = s = u + u + u + u v + v + v + v (Eq. 2.3) Arrangng he erms and groupng, a homogeneous sysem of hree equaons wh 12 unknowns ( 11 o 43 ) s obaned, as shown n equaon u + 12 u + u + 32 v v + v u X u Y u Z v X v Y v Z X = Y = Z = (Eq. 2.4)

13 I can be seen ha one sngle pon correspondence conrbues wh 3 equaons and 12 unknowns, wh only her rao s beng sgnfcan. Hence, a leas 4 noncollnear pons are needed n order o fnd he 12 parameers. However, due o he presence of nose n he measuremen, s reasonable o ransform he problem no a leas-squares-ype parameer esmaon problem, by usng more han four nosy pon correspondences. Rewrng equaon 2.4 n a marx form, equaon 2.5 s obaned, whch s he expresson of he parameer esmaon problem, ha can be solved by compung he vecor θ ha mnmses some cos funcon of he marx equaon θ = θ = 11, 12, 13, 21,..., 43. A, where [ ] T A good esmaon usng TLS echnques can be found by compung he egenvecor correspondng o he smalles egenvalue of marx A T A. u v 1 u v 1 u v 1 u u u X Y Z v v v X Y Z X Y Z = (Eq. 2.5) Wh he parameer esmaon for each srpe poson, a se of W T I s obaned. In he case of lnear scannng, a lne can be fed o he se of parameers correspondng o each marx componen Camera localzaon (self-referencng) To buld a 3D model s necessary o regser all he 3D clouds of pons acqured durng all he acquson process. The am of he regsraon process conss on ranslae he local reference sysem of each acquson sep o he world reference sysem. Ths sep s que easy when he moon of he camera beween each sep s knowed. As s presened n [Maabosh7] here are several echnques o perform hs regsraon process bu alhough he scanned objec says over he scannng surface whou any movemen, he camera moon s free. Hence our sysem s a hand-held scanner and he camera movemen beween one acquson sep and he nex one s unknown. [Héber] presens a self-referenced hand-held range sensor soluon whch consss on a rnocular sensor confguraon as s shown n fgure 2.11, where he laser projecor projecs a consan srucured lgh ono he scanned objec. These projecons are markers whch are acqured ogeher wh he scanned objec by wo cameras whch make he rangulaon o oban he 3D coordnaes. Snce he srucured lgh projeced (a leas hree pons) s known wh reference o he rao

14 beween each pon, s possble o compue he camera moon by regserng he srucured lgh acquson from one acquson o he nex or prevous one. Fgure Trnocular sensor confguraon: Two cameras and a crosshar lgh projecor. The fxed pon projecor s no depced. Ths soluon gve us an dea abou how can we relae one acquson o he nex one n order o know he camera moon. However, our sysem s formed by one camera and one laser projecor. In [Wagner7] s presened ARToolKPlus, a successor o he popular ARToolK pose rackng lbrary. Ths lbrary has been opmzed and exended for he usage on moble devces such as smarphones, PDAs and Ulra Moble PCs (UMPCs). The echnque whch s used n hs paper consss on deec a specfc marker by paern recognon and owng o he camera calbraon, he camera can be selfreferenced by knowng he poson of he marker as can be seen n fgure Fgure Basc workflow of an AR applcaon usng fducal marker rackng. From hs echnques a self-referencng camera localzaon sysem has been performed, whch s presened n Secon

15 3. Implemenaon Ths chaper presens a schedule of he seps o perform he projec, lsng deals of mplemenaon and he soluon for he handcaps whch appeared snce he projec sared. Once he decson of acheve hs projec was aken, he frs hng o do was collec all he necessary maeral such as PDA, camera and laser beam. However, hs ask ook few weeks because he maeral was scaer beween dfferen members and places of he IUT laboraory. Hence, we sared o fulfl he bographcal research n order o hnk on how o mplemen he 3D scanner, whle, on he same me, he necessay maeral was beng recolleced. The nal Gann dagram whch organse he dfferen seps o perform durng he me lne s depced n fgure 3.1. However, several asks were rescheduled, due o he reformulaon of he projec. Ths reformulaon of he projec was done durng early December afer experencng several echncal problems, such as, mpossbly o fnd a camera for he PDA or he ncompables beween he programmng ools and he operave sysem of he PDA. Handcaps whch are well dealed n he followng secons of he documen. Fgure 3.1. Gann dagram

16 The bblographcal research began specally cenred n calbraon echnques, 3D laser scanners, laser peak deecon, he prncple of rangulaon, ec. Manly, he heorecal background presened n he prevous chaper, necessary conceps o undersand before mplemenng anyhng. The hess of Dr. Fores New Mehods for Trangulaon-Based Shape Acquson usng Laser Scanners [Fores4] was a good sarng pon because nvolves several of hs heorecal conceps and accordng o Dr. Fof we mus read Acquson of mages, he nal seup Once he bblographcal research was done several algorhms such as quadrangle calbraon, laser peak deecon or marker localzaon were mplemened on Malab, because Malab allows he user o develop que fas a prooype o check f he echnque whch s beng used works or no. A hs momen, he resuls were obanng as expeced and n order o connue was necessary o receve he maeral, a leas he laser beam. The PDA s camera a hs momen was no ye avalable and was decded o connue by acqurng pcures from a moble phone, whose camera s resoluon and characerscs are more or less equal o a PDA s camera. Fgure 3.2. A Noka 628 was used nally o acqure mages. Wh a 2mpx camera we acqure mages a 64x48px Calbraon of he camera wh he complee quadrangle n Malab As has been commened before, Malab has been he envronmen n whch he 3D scanner prooype has been developed. Afer acqurng he mages of he quadrangle (Fgure 3.3) he heorecal bass of he calbraon sysem wh he complee quadrangle was verfed. Noe ha a hs me a laser beam was also provded; only he camera was mssng. A small seup n he UIT laboraory allowed o acqure es mages (Fgure 3.3) o check he Malab algorhms mplemened so far. Fgure 3.3. Quadrangle acquson used for he camera calbraon. The calbraon scrp has been developed followng he heorecal conceps showed n secon

17 The fs sep consss on denfyng he correspondences pon for each srpe poson and esmang he parameers of he 2D o 3D mappng. The acqured mage of he quadrangle (Fgure 3.3.) s presened o he user n order o denfy he quadrangle s corners (A, B, C, D ) presened n Secon (Fgure 3.4). Table 3.1 presens he Malab code for hs nalzaon: mg = mread('mage1.bmp'); % Loads he quadrangle mage fgidx=fgidx+1; fgure(fgidx); magesc(mg); pons2d = gnpu(6); Ap = [pons2d(1,:)]'; Bp = [pons2d(2,:)]'; Cp = [pons2d(3,:)]'; Dp = [pons2d(4,:)]'; Pa = [pons2d(5,:)]'; Pb = [pons2d(6,:)]'; Table 3.1. Inalzaon of he quadrangle s corners (A, B, C, D ). Noe ha a he same me ha he user esablshes he quadrangle s corners, he s asked o nroduce he lms of he laser srpe (P A and P B ) o perform he crossrao beween pons G, P R, P S and P L presened n Secon Fgure 3.4. Quadrangle s coordnaes denfcaon. Thus, lnes A -B and C -D are calculaed and s nersecon s compued n order o fnd he coordnaes of he pon G. A specfc funcon was mplemened o perform he nersecon beween wo pon s projecons. Gp = calcinersecon(ap, Bp, Dp, Cp); Table 3.2. Compuaon of he G coordnaes. A hs pon, he 3D space coordnaes of P A and P B can be compued by he use of he cross-rao and snce he 3D pons G, P R and P L are known (Fgure 2.9), he value of he cross-rao can be used n order o oban he se of 3D pons P S layng

18 on he laser srpe. Table 3.3 presens he Malab code whch performs hs calculus: % Calculae dsances for cross rao GPb = calcdsance(gp, Pb); GB = calcdsance(gp, Bp); GC = calcdsance(gp, Cp); crao = (GPb * (GB - GC)) / (GB * (GPb - GC)); % Calculae Pb n he 3D space GB1 = calcdsance(g, B); GC1 = calcdsance(g, C); GPb1 = (crao * GB1 * GC1)/((cRao-1) * GB1 + GC1); unvecorgb1 = (B - G) / calcdsance( B, G ); Pb1 = G + GPb1 * unvecorgb1; % Calculae Pa n he 3D space GA1 = calcdsance(g, A); GD1 = calcdsance(g, D); GPa1 = (crao * GA1 * GD1)/((cRao-1) * GA1 + GD1); Pa1 = [; G(2)- GPa1]; % Fnd correspomdence for addonal pons on he laser oalponsnum = 2; ponsnum = oalponsnum-1; % Image PaPb = calcdsance(pa, Pb); unvecorpapb = (Pb - Pa) / PaPb; dsancepons = PaPb / ponsnum; % Image 3d PaPb1 = calcdsance(pa1, Pb1); unvecorpapb1 = (Pb1 - Pa1) / PaPb1; dsancepons1 = PaPb1 / ponsnum; laserponsimg = zeros(2, ponsnum-1); laserponsimg1 = zeros(2, ponsnum-1); for dx=1:ponsnum-1 laserponsimg(:, dx) = (dsancepons * dx) * unvecorpapb + Pa; laserponsimg1(:, dx) = (dsancepons1 * dx) * unvecorpapb1 + Pa1; end % Creae correspondence arrays lpimg = [Pa Pb laserponsimg]; lpimg1 = [Pa1 Pb1 laserponsimg1]; lpimg1(3, :) = 1*ones(1, 1:oalPonsNum); % add he z componen Table 3.3. Compuaon of he 3D pons P S layng on he laser srpe.! I s reasonable o hnk, ha he more pons on he laser srpe, he beer he parameer esmaon of W T I wll be

19 Once all he necessary daa has been compued he calbraon marx s obaned from he equaon 2.4 as s shown n Table 3.4: % Creae he calbraon marx m = zeros(3 * oalponsnum, 12); pidx = 1; for midx=1:3:(oalponsnum*3) m(midx, :) = [lpimg(1, pidx) lpimg(2,pidx) 1 - lpimg(1,pidx)*lpimg1(1,pidx) -lpimg(2,pidx)*lpimg1(1,pidx) - lpimg1(1,pidx)]; m(midx+1, :) = [ lpimg(1, pidx) lpimg(2,pidx) 1 - lpimg(1,pidx)*lpimg1(2,pidx) -lpimg(2,pidx)*lpimg1(2,pidx) - lpimg1(2,pidx)]; m(midx+2, :) = [ lpimg(1, pidx) lpimg(2,pidx) 1 - lpimg(1,pidx)*lpimg1(3,pidx) -lpimg(2,pidx)*lpimg1(3,pidx) - lpimg1(3,pidx)]; pidx= pidx+ 1; end Table 3.4. Compuaon of he calbraon marx m.! The Malab code o oban he calbraon marx m can be found n he Malab fle: calbraon.m Tesng he calbraon marx Few mages have been acqured wh hs nal seup n order o es he mplemened Malab algorhms. Fgure 3.5. Acqured mages o check he Malab algorhms. In ha case, he calbraon marx compued n Secon 3.2 was esed by compung he Sngular Value Decomposon (SVD) of he marx calbraon m (Table 3.5). svd(x) produces a dagonal marx S, of he same dmenson as X and wh nonnegave dagonal elemens n decreasng order, and unary marces U and V so ha X = U*S*V'. Noe ha each column of V corresponds o an egen vecor of he npu marx X. These egen vecors are sored from he bgges one o he smalles. The las egen vecor, from now, s used o ransform 2D projecon pons o 3D pons n he scene

20 [u d v]= svd(m); = [ v(1,12) v(2,12) v(3,12);... v(4,12) v(5,12) v(6,12);... v(7,12) v(8,12) v(9,12);... v(1,12) v(11,12) v(12,12) ]; Table 3.5. Compuaon of. Hence, f he user eners a seres of 2D projecons, from now hese projecons can be ransformed o he 3D pons n he scene and he marx calbraon es can be performed (Table 3.5). Then, he user s asked o selec several pons belongng o he laser srpe and he 3D reconsrucon of hese pons s performed: Fgure 3.6. Laser srpe pons selecon o perform he 3D reconsrucon. % Try reconsrucon face1 = mread('face1.bmp'); fgidx=fgidx+1;fgure(fgidx); magesc(face1); face1p2d = gnpu; face1p2d = face1p2d'; face1p2d(3,:) = ones(1, sze(face1p2d,2)); face1p3d = * face1p2d; face1p3d = [face1p3d(1,:)./ face1p3d(4,:); face1p3d(2,:)./ face1p3d(4,:); face1p3d(3,:)./ face1p3d(4,:) ]; fgidx=fgidx+1;fgure(fgidx); plo3(face1p3d(1,:), face1p3d(2,:), face1p3d(3,:), 'b.'); Table D reconsrucon of he laser srpe projecon. From fgure 3.7 he 3D reconsrucon of he laser srpe can be observed whch ndcaes ha he calbraon realsed s correc

21 Fgure 3.6. Laser srpe reconsrucon. Alhough Malab vewer s no an approprae 3D vewer, from he prevous fgure can be observed he reconsrucon of he laser srpe over he face and he pons can be recognsed f he mage s compared wh he presened n fgure Projec under LabVIEW The Naonal Insrumens LabVIEW graphcal developmen envronmen helps o creae flexble and scalable desgn, conrol, and es applcaons. Afer a bref nroducon o LabVIEW by followng he uorals we realsed ha could be a good opon o perform he PDA Scanner projec. Moreover, Dr. Fof recommends and also he use of he PDA module, exenson wh whch you can creae cusom applcaons n Wndows Moble for Pocke PC devces. Hence, he projec was desgned and when was necessary o add he Camera Capure VI (Fgure 3.7) we noced ha we had a problem. The Camera Capure VI s ncorporaed n he PDA Module and saves he mages or vdeos n whaever forma he camera applcaon on he PDA arge uses. If he PDA arge does no suppor mage and/or vdeo capably, hs VI reurns an error. The problem s ha hs VI works only on Wndows Moble 5. operang sysem and he provded PDA (Axm X5 from Dell) by he lab was Wndows Moble 22 (Table 3.7a). Fgure 3.7. Camera Capure VI (LabVew). Anoher PDA (n5 Premum from Acer) was checked (Table 3.7b), hs one workng on Wndows Moble 23 SE bu sll whou workng. Thus, he soluon a hs

22 momen was fnd anoher PDA wh Wndows Moble 5. and also negraed camera, bu afer wang for few days he decson was ryng anoher envronmen o develop he projec. Processor: Inel XScale PXA25, 4Mhz Memory: 64Mb Ram memory Expanson pors: CompacFlash Type and SDIO (SD Card / MMC) Operave Sysem: Mcrosof Pocke PC 22 Screen: 24x32 pxels, colours. Table 3.7a. Dell Axs X5. Processor: Inel XScale PXA272, 52Mhz Memory: 128Mb Ram memory Expanson pors: SDIO (SD Card / MMC) and CompacFlash Type II Operave Sysem: Mcrosof Wndows Moble 23 SE Screen: 24x32 pxels, colours. Table 3.7b. Acer n5 Premum Projec under Mcrosof Vsual Sudo (C#) A hs pon was necessary o sudy dfferen language envronmens where develop he projec and Mcrosof Vsual Sudo usng as a programmable language C# was seleced. The man reason o selec hs envronmen was he SDK for Wndows Moble 23-based Pocke PCs suppled by Mcrosof. As n Secon 3.3, he frs conac wh hs envronmen conssed on desgn a small projec o nerac wh he PDA from he PC. The envronmen s nuve and frendly. I allows o run he projec drecly over he PDA whch s a good pon o perform acons such as debuggng. Thus, he projec was redesgned bu, as wh he prevous envronmen, when was necessary o acqure an mage, he same error appeared. The problem relaed wh he mage acquson from a PDA s no relaed wh he programmable envronmen, bu s relaed drecly wh he operang sysem of Fgure 3.8. Ineracon beween he PC and he PDA

23 he PDA. Under Wndows Moble 5. s no suppored he camera acquson; here are no dlls whch allow he communcaon drecly wh he PDA s camera. Then, a hs pon he only soluon was fnd a PDA wh Wndows Moble 5. or over Image acquson from he camera Fnally he camera arrved, concreely he Phoo Traveler Pocke PC camera from Veo. The maxmal resoluon of such camera s 64x48píxels (VGA) whch can be reduced o 32x24 by usng QVGA forma. A CD s provded wh drvers, whch mus be nsalled before connecng he camera no he expanson por. However, afer beng nsalled we realsed ha he camera dd no f no he expanson card of he provded PDA (Table 3.7a). Consequenly, we red wh he oher PDA (Table 3.7b) bu, afer nsallng he drvers and confgure he camera no he Acer PDA (n ha case he camera fed correcly) slled no workng. Nowadays Veo company does no exss, he offcal webse dropped ou and no suppor could be supply. The drvers are no open source and becomes a hard work and an mporan nverson of me o make he camera work. Furhermore, a descrpon n one forum was founded where s sad ha: Ths model s compable wh PAQ 36, 38 and Compaq 39; wh e57 and e74 Toshba devces and E-2 Caso Cassopea. Then, hs confrms ha hs camera could no work wh our PDAs. Whou he camera (afer several weeks) and afer check ha he operang sysem of he PDA mus be over Wndows Moble 5., we decded ogeher wh he projec supervsor o develop he projec on a convenonal compuer (lapop) by usng a webcam and he laser beam Laser deecon algorhms, mplemenaon and esng The correc localsaon of he laser srpe n an mage s a key facor n a laserbased 3D scanner, because no furher compuaon can be performed whou laser nformaon. In secon 2.3 basc prncples on laser behavour whn mages was gven, and was explaned ha he laser does no show up as a perfec smooh lne, due o several nose sources. For hs reason, s necessary o desgn and mplemen algorhms able o rereve he laser properly, hence allowng o perform furher compuaon. Two algorhms were mplemened: he frs one was esed on mages acqured wh a cell phone camera, whle wang for he fnally used camera; and he second one was desgned, mplemened and esed wh several webcams, among hem he webcam seleced for he fnal verson of he sysem Frs algorhm The frs approach uses he YCbCr colour space o perform an nal segmenaon of he mage, more precsely he Cr channel performs a clear separaon of he laser srpe from he res of he mage (Fgure 3.1b). Observng he Cr mage hsogram seems ha he laser srpe s always locaed drecly afer he mos frequen levels (Fgure 3.11a), hence he dea s o equalze he hsogram whn hs laser regon. In order o do so he boundares of he hsogram correspondng o he laser Fgure 3.9. Phoo Traveler Pocke PC camera from Veo.

24 are locaed by he means of dervave of he hsogram (Fgure 3.11b). Fxng a hreshold, he lms of he laser n he hsogram are found, and he hsogram s hen equalzed whn hese lms (Fgure 3.1c). Fnally, a medan fler s appled o elmnae he remanng nose and a fnal equalzaon of he hsogram s performed (Fgure 3.1d). a b c d Fgure 3.1. Sequence of mages obaned by he frs algorhm, from lef o rgh: a) Orgnal mage; b) Cr mage; c) Image obaned by hsogram equalzaon whn he laser boundares n he hsogram of he Cr mage; and d) Fnal mage afer medan flerng. a b c Fgure a) Hsogram of he Cr mage, n whch he laser regon s shown as a ransparen red. b) Dervave of he hsogram. c) Zoom n he laser regon, and hreshold lms for he selecon of he boundares of he laser n he hsogram Second algorhm The second approach was necessary afer observng ha he Cr channel, obaned from mages acqured wh web cams, was no separang he laser from he res. A possble explanaon for hs dfferen behavour wh respec o he phone cell camera could be he auomac exposure conrol ha mos of he web cams have ncluded n s sysem. Ths auo exposure produces enormous amouns of nose around he laser, because res o compensae he fac ha he laser s always he brghes par of he mage. Three dfferen web cams were esed and all of hem had exacly he same problem, for hs reason a compleely new algorhm was desgned. Agan, he frs sep was o compue mages n dfferen colour spaces, n hs second approach, RGB, YCbCr and HSV spaces were esed. From all hs mages, seems ha he mos useful ones are he V and Y, whch combned produce a frs esmaed of he laser regon. Tha s, he V mage conans large regon around he laser srpe (Fgure 3.12b), n whch he laser s no vsble a all, bu gves a frs approxmaon of s locaons. Ths V mages s segmened (Fgure 3.12c), and s resul s used as a mask (Fgure 3.12d) for he Y mage. From hs compuaon, he nformaon from he Y mages s consderably reduced, and so only he area around he laser s used (Fgure 3.12e), avodng oher regons ha could dffcul he laser deecon. The hsogram of hs auxlary mage shows ha he laser s locaed n he hgher nensy levels (Fgure 3.13), whch means n he far rgh par of he hsogram. Expermenally, was found ha he laser srpe appearng on one mage s composed by a ceran number of pxels, whch slghly vares from one

25 mage o anoher. Then he dea s o sar counng pxels from he las nensy value n he hsogram and go back o lower nensy values, unl hs ceran number of pxels s reached. Ths s he laser regon on he hsogram, and provdes he boundares o compue hsogram equalzaon (Fgure 3.12f). In pracce, he fnal user has he possbly o une he number of pxels, because s consderably dependen on he envronmenal condons. Once he laser s locaed, and he hsogram equalzaon has been performed a medan fler and a fnal equalzaon are appled (Fgure 3.12g). Wh he posprocessng of he mage done so far, he laser s rereved bu s wdh s more han a pxel, and n order o ge as a sngle pxel srpe, he maxmum value n each slde of he laser srpe s aken (Fgure 3.12h). a b c d e f g h Fgure a) Orgnal mage; b) V mage; c) Y mage; d) V mage segmened, he resul s a mask for he Y mage; e) Y maged afer applyng he mask from d; f) Laser srpe afer counng a ceran number of pxels wh he hghes nensy levels; g) Zoom of he laser srpe afer medan fler, s wdh s consderable; and h) Orgnal mage wh he rereved laser srpe wh a sngle pxel hckness. Fgure Hsogram of he Y mage afer applyng he mask. There s an obvous peak a. In he hgh nensy sde, he laser regon s enlghed, hs regon sars from he rgh and encompasses a fxed number of pxels, when hs number of pxels s reached, he lef boundary s gven Camera pose esmaon A mxed soluon beween he papers presened n Secon 2.6 has been done n order o know he camera poson a each me. Fxed pons n he scene are necessary n order o esmae he camera moon beween one acquson and he nex one as s sad n [Héber]. However, s no

26 possble o apply he same echnque because n our seup here s only one camera. Our soluon consss on addng a varable number of ags no he scene whch wll be dynamcally racked by he sysem and used o recognse fxed pons n he world. These feaures wll be employed o deec he camera poson and orenaon whch can be used o exrac he homographes beween he varous 3D pons rereved, whch so far are referenced o he laser plane only. The choce of an deal shape for he ag s no a sraghforward ask, here are n fac an nfne ypes of possble shapes, colours and paern ha can be used. We decded o se a number of crera whch need o be fulflled: Tag denfable from dfferen pons of vew. Independen from llumnaon changes. A sngle ag mus possess enough feaures for he esmaon of he camera poson. Small enough o be compleely vsble n feld of vew of he camera. Easy o produce. The frs aemp s shown n fgure The Harrs corner deecor s able o easly denfy he 8 corners of he ag and beng a smple shape was possble o be deeced employng he properes of s dagonals a dfferen orenaons. However here s no drec way o deec her orenaon and her sze s consderable respec o he FOV. Fgure Inal ags developed o he camera-localzaon. Fgure 3.14 shows he followng expermen. Each ag s smaller and has fve dfferen areas of dsngushable colours makng he orenaon deecable. Unforunaely, we soon dscovered ha he colours n webcam are enormously nfluenced by lghng condon and even by reflecon of nearby objecs. Fgure Coloured ags developed for camera-localzaon. The fnal ag creaed s shown fgure Insead of usng colours only hree grey levels are employed, makng exremely robus o dfferen lghng condons

27 Tag Deecon algorhm Fgure The fnal ag developed. The ag deecon algorhm developed mus be able o unquely deec all he 8 feaures n each ag whou beng rcked by oher objecs presen n he envronmen. The developmen was drven by he dea ha false posves are much more dangerous han a false negave, n fac hey lead o a compleely wrong camera pose esmaon whle a false negave wll make he sysem dscard he curren mage and wa for he followng one. Also, he algorhm works lmng as much as possble s dependence on prevous frames, allowng a successful 3D reconsrucon even on machne ha are no capable o acheve a real me frame rae. The descrpon of he algorhm follows: 1. The frs sep consss on convolvng he mage wh a dervave mask (eq. 3.1) n order o calculae he corners of he mage. 1 dx = dy = (Eq. 3.1) 2. A Harrs corner deecon s appled usng he deermnan and he consan suggesed by Peer Koves [Koves]. 3. The non maxma suppresson algorhm s appled and all he possble feaures are deeced (as shown n fgure 3.16). Fgure Feaure deecon

28 4. A wndow s open over each feaure deeced whch are sored n a ls. If here are oo many or oo few feaures for he neghbourhood ha wndow s dscarded. Our ess showed ha a range beween 8 and 15 gve good deecon resuls. 5. For each neghbourhood all he possble combnaons of 8 feaures are calculaed, hese are he ags canddaes. 6. For each ag canddae he mean s compued and used o f wo dagonals on he 8 avalable feaures. 7. If he angle beween he wo dagonals s smaller han a hreshold he ag canddae s rejeced. 8. If he dsance beween he pons and he dagonals s greaer han a hreshold he ag canddae s rejeced. 9. The nernal and exernal feaures are denfed by judgng her dsance from he mean and from he dagonal. 1. The slce wh he hghes lumnance s locaed by compung he normalzed grey value for each slce. 11. The prevous nformaon s used o calculae he vecors shown n fgure 3.17, hence denfy unambguously he ag orenaon. Fgure Deec he orenaon unquely. 12. The mean locaon of he ags n he prevous frame s employed o dsngush one ag from anoher Camera homography esmaon In order o esmae correcly he camera poson s necessary o correcly esablsh he nrnsc parameers comprsng of he dsoron marx. To do so he camera calbraon oolk from Jean-Yves Bougue [Bougue] was employed. I allowed he esmaon of he requred marx wh a back projecon error smaller han a pxel. Two dfferen echnques have been esed for he homography esmaon. The frs employs unquely affne homography o deec he no scaled homography: 1. The nal pons of he ags are sored and used as a reference. 2. For each frame n whch enough ags have been deeced he affne homography beween hem and he sored ag s esmaed wh leas mean square. 3. The homography s appled o he 3D pons on he laser plane

29 The example of he resul s shown n fgure I should be noced ha he reconsrucon obaned does no have he correc scale because assumes ha all he ags s a nfne. For hs reason anoher echnque was explored. Fgure Regsered 3D pons wh affne homographes. In order o oban he reconsrucon wh he correc scale we decded o employ rangulaon. In fac, our sysem can be consdered as a bnocular sysem n whch one camera s fxed (he vew of he reference pon) and he second one s moved frame by frame, hence needs o be connuously calbraed n order o rereve s poson respec o he frs one. Ths s possble because he measures of he ags are known. The seps of he algorhm follows: 1. The nal pons of he ags are sored and used as a reference. 2. The fundamenal marx beween he curren vew and he reference vew s calculaed employng he normalsed 8 pon algorhm gven by Harley and Zsserman [Harley, Zsserman]. 3. The essenal marx s exraced from he fundamenal marx employng he known nrnsc parameers. 4. Sngular Value Decomposon s used o exrac he four possble soluons o he camera pose n he exrnsc marx. 5. The correc soluon s seleced usng he echnque llusraed by Davd Nsér [Nsér] whch s an effcen way o rule ou all he no feasble soluons. 6. The 3D pons are regsered usng he camera pose calculaed. 7. The pons are scaled by comparng he curren values wh he real sze of he ags

30 Fnal Implemenaon Afer developng varous prooype wh he MATLAB envronmen, he real sysem was mplemened n C++ n order o be able o execue all he algorhms close o real me, and hs way demonsrang ha he scanner s effecvely usable. The OPENCV lbrary was used o speed up he developmen and o provde a mulplaform access o he webcam. Smlarly, he GTK+ was seleced as he GUI lbrary because of s porably hroughou varous plaforms and s clean objec orened nerface. Fgure 3.19 shows a screensho of he sofware durng a scannng sesson. In he wndow s possble o see he real me oupu of some of he algorhms ha run under he hood (.e. mage correcon, laser and ag deecon). The frs wo parameers allow o change he sengs of he laser deecon algorhm n order o have a relable oupu n all lgh condons. The las parameers are he number of ags ha he sysem should expec o rereve he 3D poson of he laser pons. Ths makes much more ressan n suaons when ags are emporary los because of mage nose. The sysem also mplemens he calbraon of he 3D pons esmaor wh he complee quadrangle. Fgure The sofware n acon. In he curren mplemenaon he 3D pose esmaon s no drecly compued. Insead all he 3D laser pons and ag posons are sored n a ex fle whch can be analysed a a laer sage. The reason s ha 3D regsraon algorhm was no horoughly esed and a prooype ha gves sasfacory resuls was no developed

31 4. Conclusons Wha sared as a scanner for a PDA for varous reasons evolved no a webcambased scanner whch uses only cheap hardware ha can be bough n any hgh sree shop. I was shown how was possble rereve 3D pons n varous lumnance condons whou any laser fler. Also, a echnque o unambguously deec small paper ags a varous orenaons, scales and llumnaons was presened. They can be used o deec he camera pose, whch n urn can be employed o regser all he 3D pons no he same reference sysem. Gven he me consrans was no possble a horough verfcaon and valdaon of he cloud of 3D pons produced. A par from hs aspec, s lef as a furher work he mplemenaon of a vsualzer of he 3D model n real me and moreover some quanave comparson beween hs scanner and a commercal one. References Merology. hp://en.wkpeda.org/wk/merology Conac 3D scanners: hp:// [Héber] Parck Héber. A Self-Referenced Hand-Held Range Senso.Compuer Vson and Sysems Laboraory Laval Unversy, Québec, Canada. [Wagner7] Danel Wagner and Deer Schmalseg. ARToolKPlus for Pose Trackng on Moble Devces. Insure for Compuer Graphcs and Vson, Graz Unversy of Technology. Compuer Vson Wner Workshop 27. [Fala4] Fala, M. NRC-CNRC. ARTag Revson 1. A Fducal Marker Sysem Usng Dgal Technques. Naonal Research Councl Canada. Insue for Informaon Technology. [Fores4] J. Fores. PhD Thess. New mehods for rangulaon-based shape acquson usng laser scanners. Unversa de Grona. Deparamen d Elecrònca, Informàca Auomàca. [Maabosch7] C. Maabosch. PhD Thess. Hand-held 3D-Scanner for large surface regsraon. Unversa de Grona. Deparmen of Elecroncs, Compuer Scence and Auomac Conrol. [Chen87] C.H. Chen and A.C. Kak. Modelng and calbraon of a srucured lgh scanner for 3-d robo vson. In Proceedngs of he IEEE conference on robocs and auomaon, pages IEEE, [Koves] Peer Koves. NONMAXSUPPTS - Non-maxmal suppresson for feaures/corners. School of Compuer Scence & Sofware Engneerng. The Unversy of Wesern Ausrala. Copyrgh (c) hp:// [Bougue] Jean-Yves Bougue. Camera Calbraon Toolbox for Malab. hp://