3D RECONSTRUCTION USING A SPHERICAL SPIRAL SCAN CAMERA

3D RECONSTRUCTION USING A SPHERICAL SPIRAL SCAN CAMERA A Thesis Submitted to the Gaduate Schoo of Engineeing and Sciences of zmi Institute of Technoogy in Patia Fufiment of the Requiements fo the Degee of MASTER OF SCIENCE in Compute Engineeing by Mustafa VATANSEVER Octobe 26 ZMR

We appove the thesis of Mustafa VATANSEVER Date of Signatue.................................... 1 Octobe 26 Pof. D. Sıtkı AYTAÇ Supeviso Depatment of Compute Engineeing zmi Institute of Technoogy.................................... 1 Octobe 26 Asst. Pof. D. evket GÜMÜTEKN Co-Supeviso Depatment of Eectica and Eectonics Engineeing zmi Institute of Technoogy..................................... 1 Octobe 26 Pof. D. Hais PÜSKÜLCÜ Depatment of Compute Engineeing zmi Institute of Technoogy..................................... 1 Octobe 26 Asst. Pof. D. Zekeiya TÜFEKÇ Depatment of Eectica and Eectonics Engineeing zmi Institute of Technoogy..................................... 1 Octobe 26 Pof. D. Eo UYAR Depatment of Mechanica Engineeing Dokuz Eyü Univesity..................................... 1 Octobe 26 Pof. D. Kayhan ERCYE Head of Depatment Depatment of Compute Engineeing zmi Institute of Technoogy.................................... Assoc. Pof. Semahat ÖZDEMR Head of the Gaduate Schoo

ACKNOWLEDGEMENTS I oud ike to thank to my supeviso Pof. D. Sıtkı Aytaç fo his guidance and suppot. I oud ike to thank to my co-supeviso Asst. Pof. D. evket Gümütekin fo his guidance, ideas and invauabe suppot on soving the technica detais. I aso oud ike to thank to Pof. D. Hais Püskücü, Asst. Pof. D. Zekeiya Tüfekçi, Pof. D. Eo Uya and Inst. Toga Ayav fo thei advice in evising the thesis. I am vey gatefu fo my famiy and fiancee fo thei continuos suppot.

ABSTRACT 3D RECONSTRUCTION USING A SPHERICAL SPIRAL SCAN CAMERA Constuction of 3D modes epesenting industia poducts/objects is commony used as a peiminay step of poduction pocess. These modes ae epesented by a set of points hich can be combined by pana patches (i.e. tianguation) o by smooth suface appoximtions. In some cases, e may need to constuct the 3D modes of ea objects. This pobem is knon as the 3D econstuction pobem hich is one of the most impotant pobems in the fied of compute vision. In this thesis, a sytem has been deveoped to tansfom images of ea objects into thei 3D modes automaticay. The system consists of a PC, an inexpensive camea and an eectomechanica component. The camea attached to this component moves aound the object ove a spia tajectoy and obseves it fom diffeent vie anges. At the same time, featue points of object suface ae tacked using a tacking agoithm ove object images. Then tacked points ae econstucted in 3D space using a steeo vision technique. A suface appoximation is fitted to this 3D point set as a ast step in the pocess. Open souce Inte -OpenCV C ibay is used both in image captuing and in image pocessing. iv

ÖZET KÜRESEL SPRAL TARAYICI KAMERA LE 3B GERÇATIM Endüstiye üünein/nesneein 3B modeeinin outuuması geneike üetim süecinin ik adımı oaak kuanımaktadı. Bu modee noktaa kümesiye gösteii ve bu noktaa üçgen gibi düzemse eemanaa veya düzgün yüzey yakaımaıya bietiii. Ancak bazı duumada geçek nesneein 3B modeeini outumamız geekebii. Bu pobem 3B geiçatım pobemi oaak biini ki; bu pobem bigisayaa göme aanının en önemi pobemeinden biidi. Bu tezde geçek nesneein göüntüeinden 3B modeeine otomatik geçii saayan bi 3B geiçatım sistem geitiimiti. Sözkonusu sistem bi PC, düük maiyeti bi kamea ve eektomekanik bi paçadan oumaktadı. Eektomekanik paçaya baı kamea nesne etafında spia bi yöüngede haeket ettiieek nesneyi deiik açıadan gömesi saanmıtı. Aynı zamanda bi takip agoitmasıya, nesne yüzeyinde beieyici niteii oan noktaaın, nesne göüntüei üzeindeki takibi yapımıtı. Bi ikii göme teknii kuanıaak takip edien noktaaın 3B geiçatımı yapımıtı. Son adım oaak geiçatımı yapımı oan 3B nokta kümesinden bi yüzey yakaımı ede edimiti. Kameadan göüntü aımı ve ienmesinde açık kaynak kodu Inte -OpenCV C kütüphanesi kuanımıtı. v

TABLE OF CONTENTS LIST OF FIGURES... vii CHAPTER 1. INTRODUCTION... 1 CHAPTER 2. CAMERA CALIBRATION... 4 2.1. Pinhoe Camea Mode (Pespective Pojection Mode)... 4 2.2. Intinsic Camea Caibation... 6 2.2.1. Intinsic Camea Caibation Using Matab Camea Caibation Toobox... 9 2.3. Extinsic Camea Caibation... 1 CHAPTER 3. 3D RECONSTRUCTION FROM MULTIPLE VIEWS... 14 3.1. Singe Vie Geomety... 14 3.2. To Vie Geomety (Steeo Visio)... 15 CHAPTER 4. EXPERIMENTAL SETUP: SPHERICAL SPIRAL SCAN MECHANISM... 17 4.1. Steppe Motos... 21 CHAPTER 5. 3D RECONSTRUCTION ALGORITHM... 24 5.1. Featue Point Detection... 27 5.2. Tacking of Featue Points... 3 5.3. 3D Reconstuction of Tacked Featue Points... 35 CHAPTER 6. SURFACE FITTING... 42 CHAPTER 7. CONCLUSIONS... 46 BIBLIOGRAPHY... 48 APPENDIX A. COMPUTER SOFTWARE... 5 vi

LIST OF FIGURES Figue Page Figue 1.1. Genea system setup iustation... 2 Figue 2.1. Pinhoe camea mode... 4 Figue 2.2. Side vie of pinhoe camea mode... 5 Figue 2.3. Scaing facto... 6 Figue 2.4. Sce facto... 7 Figue 2.5. Radia (Bae) distotion... 7 Figue 2.6. Images of same scene taken ith a shot foca ength camea (a) and ong foca ength camea (b)... 9 Figue 2.7. A vie of caibation patten... 9 Figue 2.8. (a) An image befoe undistot pocess (b) afte undistot pocess... 1 Figue 2.9. Position and oientation of pinhoe camea in od coodinates... 11 Figue 2.1. A 2D vecto v in to diffeent efeence sytems ( XY and X Y )... 11 Figue 3.1. Missing of depth hie pojecting... 14 Figue 3.2. Steeo vision... 15 Figue 3.3. Cossing ines in steeo vision... 16 Figue 4.1. Iustation of expeimeta setup used in this thesis... 17 Figue 4.2. Isometic vie of hemispheica spia... 18 Figue 4.3. A 2-tun spheica spia, od and camea coodinate systems used in the thesis. (a) side vie and (b) top vie... 19 Figue 4.4. Rotation ange-time gaph in spheica spia scan pocess... 2 Figue 4.5. A vie of expeimenta setup... 2 Figue 4.6. Steppe moto... 21 Figue 4.7. Discete spheica spia fomed by steppe motos... 22 Figue 4.8. Image sequence of a test object obseved by the camea... 23 Figue 5.1. Fo chat of pesented 3D econstuction agoithm... 25 Figue 5.2. Test objects used in this thesis fo 3D econstuction. (a) a box (b) a piece of egg caton... 29 vii

Figue 5.3. Featue point detected images of test objects (a) box image ith 118 detected featue points (b) egg caton ith 97 detected featue points... 3 Figue 5.4. A otating sphee fom eft to ight and coesponding optica fo fied... 31 Figue 5.5. Featue point tacking ove the box test object... 33 Figue 5.6. Featue point tacking ove the egg caton test object... 34 Figue 5.7. The vecto defining the diection of econstuction ines... 37 Figue 5.8. Reconstucted 3D point couds of test objects. (a) box (b) egg caton... 4 Figue 5.9. Reconstucted points of the box using non unifom ange... 41 Figue 6.1. A meshed hemisphee ith 2x15 Intevas... 43 Figue 6.2. A synthetic data set ith 54 points (a) pojected data points on to hemisphee (b) and fitted suface on to data points ith mesh size (25x2)... 44 Figue 6.3. Point couds ith coesponding sufaces... 44 Figue 6.4. Egg caton suface. Fitted using the Mathematica softae... 45 viii

CHAPTER 1 INTRODUCTION 3D econstuction is a tansfomation fom 2D images of ea objects o scenes to 3D compute modes hich is taditionay an impotant fied of compute vision. Afte yeas of theoetica deveopment, in ecent yeas the inteest in 3D econstuction systems have been apidy going. Such systems ae being impemented fo many appications in pactice such as industia vision, vitua eaity, compute aided design (CAD), medica visuaization, movie effects, game industy, advetising, e-commece etc. 3D econstuction usuay consists of to steps. Fist step is acquiing 3D points fom scenes that can be done by ange scannes o mutipe vie systems. The techniques fo acquiing 3D points fom mutipe vies ae idey investigated by eseaches due to thei o cost and ea-time pefomance. The output of such systems is usuay unoganized 3D point couds. These points ae used in the second step hich invoves fitting a mathematica mode and getting paameteized sufaces that fom a 3D compute mode. Without any pe-defined infomation about obseved scene; it is vey had to obtain usefu 3D modes in pactice. To simpify the pobem some assumptions on the objects and envionments (such as camea geomety, backgound, ighting, e textued sufaces etc.) ae used in this thesis. In (Sainz et a. 22), authos epesent a voxe-caving agoithm to obtain a 3D voxe-based suface mode fom mutipe vies, hich uses uncaibated cameas. Whie (Müayim et a.23) and (Wiiams 23) use singe axis tun tabe move to econstuct sihouette based modes. These methods invove caving a voxeized piece of vitua mateia that contains the objects. Daback of such system is extensive computation time. (Sainz et a. 22) pesents a method to impove un-time speed of the space caving agoithms. In (Siddiqui et a. 22a) and (Siddiqui et a. 22b) a method is descibed fo 3D suface econstuction fom mutipe vies. It uses given featue point coespondences as input and takes the advantage of pojective invaiance popeties of ationa B-spines to econstuct 3D ationa B-spine suface mode of the object. 1

In this study, a nove method fo 3D econstuction using a pedetemined camea and object motion is pesented. This method eies on an eectomechanica system and some assumptions on the envionment. Backgound is assumed to be hite, ighting is assumed to be constant, object is assumed to be e textued to be abe to find good featues fo tacking. Since the fina suface mode consists of appoximation of 3D featue points, a high numbe of featue points is needed. To get moe featue points, the camea shoud be abe to scan the object fom sevea viepoints. Spheica spia tajectoy of camea aound object is poposed hee to each diffeent vies fom diffeent sides of the object (Nakbuakae et a. 25). This is the main advantage of the this study fom simia 1 axis (cyindiica) tun tabe setups (Müayim et a. 23), (Femont et a. 24), (Yemez et a. 24), (Sainz et a. 24). These methods eithe econstuct shape fom sihouette o fom steeo. Thei common daback is the imited vie anges. Anothe consideabe contibution of this thesis is the use of optica fo to tack featue points to eiminate the point coespondence pobem. Instead of moving the camea in 2 degees of feedom (DOF) to get spheica spia scan (SSS); 1 DOF camea move and 1 DOF object move is used to get the same effect. SSS is detaied in Chapte 4. Genea sytem setup iustation can be seen in Figue 1.1. Figue 1.1. Genea system setup iustation. The main focus of the thesis is; ecoveing and tacking some 2D featue points on the images of the objects hie object and camea ae otated in contoed anges (aong spheica spia tajectoy) by steppe motos and cacuating 3D coodinates of the featue points by using steeo vision geomety. 2

Afte competing SSS, 3D positions of featue points ae detemined. Then suface fitting agoithm is appied to the 3D featue points. This thesis is oganized as foos: Chapte 2 gives a bief ovevie of camea caibation. Chapte 3 gives the backgound fo the use of steeo vision geomety fo acquiing depth infomation. The deveoped system setup is descibed in Chapte 4. The 3D point acquiing agoithm is expained in Chapte 5. Fitting suface to point couds is the subject of Chapte 6. Finay the thesis ends ith a concusion in Chapte 8. 3

CHAPTER 2 CAMERA CALIBRATION Camea caibation can be descibed as the pocess of detemining intinsic paametes (camea cente, foca ength, ske facto, adia distotion coefficients) and extinsic pametes (otation and tansation fom od coodinates) of the camea. In Section 2.1 pinhoe camea mode is descibed biefy. Section 2.2 and Section 2.3 give bief descibtion about intinsic and extinsic camea caibation espectivey. 2.1. Pinhoe Camea Mode (Pespective Pojection Mode) In pinhoe camea mode, a ays pass fom a point caed optica cente (C) and intesects ith the image pane at a distance f (foca ength). Fo simicity the image pane is consideed to be in font of camea cente instead of back (ea image pane). This is caed vitua image pane and an iustation of pinhoe camea mode ith vitua image pane can be seen in Figue 2.1. In pinhoe camea mode, images ae fomed on image pane via pespective pojection. Figue 2.1. Pinhoe camea mode. (Souce: WEB_1) 4

Figue 2.2. Side vie of pinhoe camea mode. (Souce: (WEB_1) Consideing any 3D point A(x,y,z) in font of the camea, its image coodinates a( x i, y i ) i be; x i f x z y i f y z These equations can be easy obtained by using geometica simiaities of tianges (see Figue 2.2). This tansfomation can be epesented by the fooing matix equation: u f v f 1 x y z 1 (2.1) Whee x i u y i v (u, v, ) ae the homogeneous coodinates of the point 5

6 2.2. Intinsic Camea Caibation Rea cameas ae not exacty same as the simpified camea modes (e.g Pinhoe camea mode, see Figue 2.1) they have some fauts due to physica imitations and the poduction pocess. These fauts ae biefy given beo. Image cente. Usuay pincipa axis does not passes though cente of image pane. In the idea case image cente is at (idth/2) u, (height/2) v. But in eaity, this is not the case. We shoud take into account the offset fom the idea cente, the pojection matix becomes; 1 1 z y x v f u f v u (2.2) Whee u and v ae optica cente in u and v diection espectivey. Figue 2.3. Scaing facto. Diffeent scaing factos fo u and v diection (e.g. as in Figue 2.3) is necessay especiay hen the CCD sensos have ectangua geomety. The pojection matix can be modified as fooing to ceate this effect: 1 1 z y x v u v u β α (2.3)

hee α k x f and β k y f. Hee k x and k y ae diffeent scaing coefficients in u and v diections espectivey. Figue 2.4. Ske facto. Ske facto. This is the faut that the ange beteen u and v diection is not 9º (see Figue 2.4). A sma modification in the pojection matix can coect this effect. u α v γ β u v 1 x y z 1 (2.4) Figue 2.5. Radia (Bae) distotion. Radia distotion is caused by impefections in ens design. Radia distotion makes staight ines seem cuved in image pane (see Figue 2.5 and Figue 2.6a). Radia distotion is a non inea defomation hich can not be handed by modifying the pojection matix. It is usuay handed by appoximating the defomation by a poynomia; 2 3 + a + b + 7

hee (, ) ae the (idea, distoted) distance of an image point to cente and a, b ae the paametes of the tansfom. Usuay, the highe degees of the poynomia ae ignoed. If e e-aange the 3x4 pojection matix given in equation 2.4 e get; α γ β u v 1 α γ β u1 v 1 1 1 K [ I ] 3 3 (2.5) In equation 2.5 K is the camea intinsic matix. Fo metic econstuction; camea cente, foca ength, ske facto and adia distotion coefficients have to be knon and have to be coected befoe pocessing the images. Camea intinsic paametes ae usuay given in camea intinsic matix as foos: K α γ u v 1 β (2.6) In equation 2.6 u and v ae cooidinates of pincipa point in u and v diections espectivey, α and β ae pixe unit foca engths in u and v diections espectivey, γ is skeness facto. Radia distotion paametes a and b ae aso consideed as intinsic camea paametes. In this pape e do not focus on soving the caibation paametes. Fo futhe infomation about camea caibation, eades may efe to (Zhang 1999) and (Zhang 2). Matab Camea Caibation Toobox (based on (Zhang 1999) ) is used hee to compute the intinsic paametes of the camea. Then camea intinsic matix is manuay passed to OpenCV fo futhe steps. 8

Figue 2.6. Images of same scene taken ith a shot foca ength camea ith sevee bae distotion (a) and ong foca ength camea ith o distotion (b). (Souce: Hatey et a. 22) 2.2.1. Intinsic Camea Caibation Using Matab Camea Caibation Toobox Matab Camea Caibation Toobox is based on (Zhang 1999). The toobox uses at east to vies of a pana patten taken fom diffeent oientations. The patten metics have to be knon and enteed to the toobox befoe caibation pocess. Such a patten consists of 7x9 squaes each of hich has the size of 3x3 miimetes. The caibation patten used in this thesis can be seen in Figue 2.7. The use has to cick on coesponding cone points at diffeent vies. These points ae used fo caibation. The moe vies ae used in pocess, moe sensitive caibation paametes ae extacted. In this study 5 images of caibation patten ae used. One of these images can be seen in Figue 2.7. Figue 2.7. A vie of the caibation patten. 9

In equation 2.7 Matab intinsic paamete esut fo ou camea (an inexpensive USB camea) is shon. γ in the matix that means ange beteen u and v is 9º 445.89271 14.14484 K 447.4232 115.21561 (2.7) 1 Besides, adia distotion coefficients cacuated by the toobox ae: Kc [ -.3414.42971 -.17 -.684 ] (2.8) The toobox can aso undistot the images using camea intinsic paametes. An exampe of an image befoe and afte undistotion pocess is shon in Figue 2.8. Radia distotion effect can be seen in Figue 2.8 (a). The bended tabe edge can be easiy seen in the image caused by adia distotion. The same image afte undistoting ith staight tabe edge can be seen in Figue 2.8 (b). (a) (b) Figue 2.8. (a) An image befoe undistot pocess (b) afte undistot pocess. (Souce: Hatey et a. 22) 2.3. Extinsic Camea Caibation Extinsic camea paametes descibe 3D position and oientation of the camea in od coodinates. These paametes ae given in a otation matix and a tansation 1

matix. These paametes ae combined in a singe tansfom matix. Then any 3D point can be conveted fom od coodinates to camea coodinates using tansfom matix. Once e get the coodinates of 3D points in camea coodinates, then e can easiy find 2D image coodinates of those points using pojection matix descibed in Section 2.1 and Section 2.2. Figue 2.9. Position and oientation of pinhoe camea in od coodinates. (Souce: Hatey et a. 22) Fo a 2D vecto v, efeence system otation about Z axis, equations can be given as beo (see Figue 2.1): Figue 2.1 A 2D Vecto v in to diffeent efeence sytems ( XY and X Y ). x x. cos( α) + y.sin( α) y y. cos( α) x.sin( α) 11

12 In matix fom; y x y x ) cos( ) sin( ) sin( ) cos( α α α α If e expand it to 3D; otation about Z axis can be given as beo: ).sin( ) cos(. α α y x x + ).sin( ) cos(. α α x y y z z In matix fom; z y x z y x 1 ) cos( ) sin( ) sin( ) cos( α α α α Then otation matix aound z axis is: 1 ) cos( ) sin( ) sin( ) cos( ) ( α α α α α z R (2.9) When e extend equation 2.9 to otation about y and x axises the fu otation matix (R) becomes: (2.1) Rz Ry Rx ) cos( ) sin( ) sin( ) cos( 1 cos sin 1 sin cos 1 ) cos( ) sin( ) sin( ) cos( φ φ φ φ β β β β α α α α R

Tansation matix is given as foos: 1 X 1 Y T (2.11) 1 Z 1 hee X, Y and Z ae ne od coodinates of the camea. In this study e used steppe motos to otate the camea and objects so e kno the otation anges aound x-y-z (φ, β,α ) in each step. We aso used a fixed adius od to pace the camea. So e can easiy cacuate 3D position of the camea in od coodinates using otation anges and fixed adius (R). R is the distance fom pojection cente of the camea to od efeence point (,,). Detais about expeimenta setup and the cacuation of otation and oientation ae given in Chapte 4. 13

CHAPTER 3 3D RECONSTRUCTION FROM MULTIPLE VIEWS 3.1. Singe Vie Geomety 3D econstuction is the pobem of acquiing 3D points of scenes o objects fom thei 2D image vies. Looking at a singe 2D image, the depth (3d dimention) is not diecty accesibe even fo a human. This can be expained easiy; supposing that thee is a ight ay (L) fom ea point P to image pane that passes though optica cente (pespective pojection). p is the pespective pojection of point P on to image pane. The point p(u,v) on the image pane, hich is the pojection of a singe point P(x,y,z) does not cay any infomation on hee P is ocated aong the ay (L). See Figue 3.1. Figue 3.1. Missing of depth hie pojecting. As can be seen in Figue 3.1, pojected coodinates of 3D points A,P and B ae (u,v). Athough 3D coodinates ae diffeent, thei image coodinates ae same. 3D econstuction is the invese pocess of pojection. Given 2D coodinates of a point p, e have to detemine hich 3D point coesponds to p. It is cea that a singe vie is insufficient fo this decision. Moe vies ae needed. 14

3.2. To Vie Geomety (Steeo Vision) Assume that thee ae to fuy caibated cameas (i.e. camea centes, foca engths, pixe atios, skeness, adia distotion coeffitions, 3d positions and oientations in od coodiantes ae knon) as iustated in Figue 3.2. Both cameas ae obseving the same scene/object. Suppose that thee is a point (P) in the scene visibe in both eft and ight cameas. Let (p) be the pojection of (P) onto eft camea s image pane and (p) be the pojection of (P) onto ight camea s image pane. Since image coodinates of (p), (p) and camea paametes ae knon, (P) can be econstucted as intesection of to ight ays (L) and (L). Figue 3.2. Steeo vision. In pactice, to ays i not intesect due to caibation, featue ocaization and featue tacking eos (see Figue 3.3). Hoeve, the 3D position of P(x,y,z) can sti be estimated. The appoach used in this study is to constuct a ine segment pependicua to the to ays hee to ays ae cosest to each othe. And then the mid point of the ine segment is used as the econstucted point. (Fosyth et a.23). When this ine segment is onge than a pedetemined theshoding vaue; the econstucted point is detemined as ong econstucted point. And the ong econstucted points ae not ecoded as econstucted points. In pactice, the biggest pobem of steeo vision is coespondence pobem. Coespondence pobem is the pobem of finding hich point on eft image matches hich point on ight image o vice vesa. Thee ae diffeent soutions fo coespondence pobem in iteatue. In this thesis, a diffeent technique is used to dea 15

ith coespondence pobem. That is hy e do not focus on soving the cosepondence pobem in this thesis. The eades ho ae inteested in soving the coespondence pobem diected to (Fosyth et a.23) and (Hatey et a 22). In this thesis, a moving camea and moving object ae used to fom steeo vision. An optica fo based tacking agoithm (Lucas et a. 1981) is used to avoid coespondence pobem. The tacking agoithm enabes us to tack the featue points in each vie, so e kno coesponding 2D image coodinates of featue points. Since steppe moto diven objects /scenes featue points dispacement ae sma enough in images, tacking agoithm easiy tacks featue points ithout osing them. The OpenCV impementation of Lucas&Kanade Pyamidia Tacking Agoithm is used in this study. Figue 3.3. Cossing ines in steeo vision. The eaistic mode iustated in Figue 3.3 is used in this study fo 3D econstuction Given camea paametes and featue coespondence in to vies, the goa is to econstuct 3D positions of featue points. One shoud keep in mind that intinsic camea paametes ae pe-cacuated fo ate use, and extinsic paametes ae cacuated once in a fe steps of steppe motos. Extinsic paametes (tansation and oientation) of camea ae easiy cacuated using otation anges of steppe motos. Cacuation of camea s 3D position and oientation i be expained in next chapte. Hee e i assume that intinsic and extinsic paametes ae knon, and e i just focus on ho to econstuct 3D point using 3D geomety. Reconstuction fom moe than to vies is not disccused in this thesis. 16

CHAPTER 4 EXPERIMENTAL SETUP: SPHERICAL SPIRAL SCAN MECHANISM Cassica tun-tabe based 3D econstuction appoaches in steeo vision usuay uses singe axis otation tabes (Müayim et a. 23), (Wiiams 23), (Femont, V., and Cheai R. 24). Singe axis otation tabes gives imited vie of the object. In this study e deveoped a 2 degee of feedom (DOF) system to get moe vies of the object. Ou 2DOF setup moves the camea in a hemispheica spia tajectoy. Camea s optica axis is oiented to obseve the object(s) hie it is moving in spheica tajectoy. Figue 4.1. Iustation of expeimeta setup used in this thesis. The spheica and catesian coodinates can be eated by: x sin( β ) cos( α) y sin( β )sin( α) z cos( β ) (4.1) 17

If e extend the unit sphee to R adius sphee; equations in 4.1 become: x R sin( β ) cos( α) y R sin( β ) sin( α) z R cos( β ) (4.2) hee R is distance fom camea optica cente to od coodinate system oigin in this study (see Figue 4.2). Figue 4.2. Isometic vie of hemispheica spia. 18

(a) (b) Figue 4.3. A 2-tun spheica spia, od and camea coodinate systems used in the thesis. (a) side vie and (b) top vie. Expeimenta setup used in this thesis consists of to steppe motos, to steppe moto dives contoed by PC paae pot and a USB camea. One of the steppe motos is paced veticay and the othe is paced hoizontay so that thei axises ae pependicua to each othe. The intesection point of these axises ( see Figue 19

1.1 and Figue 4.3) is accepted as the oigin of the od coodinate system (WCS) (X- Y-Z). Z-axis is chosen to be aong vetica moto axis and Y-axis is chosen to be aong hoizonta moto axis. Fom no on e i ca vetica moto as z-moto because it otates the object aound Z axis. And e i ca hoizonta moto as y-moto because it otates the camea aound Y axis. Camea coodiante system (CCS) (i-j-k) is iustated in Figue 4.3. Camea optica axis is aays aigned ith the ine that passes though the oigin of CCS and the oigin of WCS. Figue 4.4. shos change of z and y moto otation ange (α, β ) in time t hie SSS is in pocess. Figue 4.4. Rotation ange-time gaph in spheica spia scan pocess. Figue 4.5. A vie of expeimenta setup. 2

4.1. Steppe Motos Steppe motos ae eectomechanica devices that convets eectica puses to shaft (oto) otations. A steppe moto consists of a static body (stato) and otating shaft (oto). The otos have pemanent magnets attached to them. Aound the statos thee ae seies of cois that ceate magnetic fied. When moto is enegyzed, pemanent magents and cois inteacts ith each othe. If cois ae enegized in a sequence, the oto otates foad o backads. Figue 4.6 shos a typica steppe moto. Figue 4.6. Steppe moto. Steppe moto otos otate in cetain anges that is dependent on numbe of phases and numbe of oto teeth they have. And otation ange pe step is cacuated ith the fomua given beo, 36 θ s (4.3) m.n hee m is the numbe of phases and N is the numbe of oto poes (See Figue 4.6). Both z-moto and y-moto used in this study has 4 phases and 5 oto teeth so they have 36 4.5 θ s 1.8 (4.4) step degee. In ou expeiments each step coesponds to steps fo 1.8.Each moto needs 2 36 otation. To compete 2-tun SSS pocess in t time (See Figue 4.3), z- 21

moto has to otate 72 (o 4 steps) and in the same time y-moto has to otate (o 5 steps). That means y-moto has to otate one step ( 1.8 ) in eachs 8 steps ( 14.4 ) of z-moto. When e use this infomation in the coodinate tansfomations (Eq.4.2), the spheica spia tajectoy can be computed as shon in Figue 4.7. 9 (a) (b) (c) Figue 4.7. Discete spheica spia fomed by steppe motos. In Figue 4.7 thee ae 4 points that cove 4* 1.8 72 (2-tun spheica spia). That means α stats fom deceases 1.8 in each 14.4 step of α. and inceases to 72. And β stats fom Steppe motos can aso be diven in haf step mode ith a 9 and.9 step ange. But 1.8 is enough fo tacking agoithm not to ose tacking in this study, so haf step dive is not consideed to be necessay. 22

Figue 4.8 shos an image sequence of a test object obseved by the camea ove the spheica spia tajectoy. Figue 4.8. Image sequence of a test object obseved by the camea. 23

CHAPTER 5 3D RECONSTRUCTION ALGORITHM A fo chat of the poposed 3D econstuction agoithm can be seen in Figue 1.3. The agoithm is buit on Inte s OpenCV ibay to be abe to use many peimpemented codes, such as image captuing function cvcaptuefomcam(), undistoting function cvundistotonce(), featue point detection function cvgoodfeatuestotack(), optica fo based function fo featue point tacking cvcacopticafopylk() etc. In this study e captued 32x24 pixe images ith 24 bit coo depth in RGB ith an inexpensive USB camea. Since coo is not competey necessay in this appication, e conveted the images to 8 bit depth gay-scae images. Afte an initia camea caibation step, the fooing pocedue is impemented. Reconstuction pocess stats at the side of the hemispheica spia ( α and β 9 ) and it ends in end point of the spia ( α 72 and β ) (See Figue 4.2 and Figue 4.7). Consideing Figue 4.7 it can be seen that the econstuction agoithm consists of 4 step. In each step, a ne image is captued, featue points ae detected on the image o tacked (if aeady detected). Once in evey 1 steps (18 ) 3D econstuction is done using the agoithm expained in Section 3.2 and featue points ae ceaed. Ne featue points ae edetected on the image, z-moto and y-moto (so the camea) ae diven to next positions via paae pot of the compute. The biggest pobem of econstuction once in evey 1 steps (18 ) is that; sensitivity of econstuction is deceasing hen camea optica axis come nea paae to Z axis of WCS. It is because hen the ange beteen to camea s optica axis is decease; the position of econstuctued point change in geat vaue coesponding to sma ange changes. To aeviate this pobem e used an unifom ange in 3D to make sue that ange beteen to optica axises ae not ess than some vaue (i.e. 18 ). On the othe hand once in 1 step econstuction has unifom ange in 2D. In Figue 5.9 incoect econstucted points can be seen on the box object. 24

Figue 5.1. Fo chat of pesented 3D econstuction agoithm. 25

1. α 2. β 9 3. get ne image fom the camea (I1) 4. undistot the (I1) image (using intinsic paametes of camea) 5. detect featue points on the image 6. stoe image coodinates of featue points on memoy (L) 7. fo step1 to 4 ; stepstep+1 8. get ne image fom the camea (I2) 9. undistot the I2 image 1. tack featue points on I1 and I2 using optica fo 11. stoe ne image coodinates of featue points on memoy (R) 12. if (step mod 1 ) 13. 3d econstuction using stoed image coodiantes (L and R) (See Section 3.2) 14. eset featue points (cea a featue points) 15. detect featue points on the image (ne featue points ae detected on the same image) 16. stoe featue point image coodinates on memoy (L) 17. end if 18. otate z-moto one step foad 19. α step *1.8 (o α α +1.8 ) 2. if (step mod 8 ) 21. otate y-moto one step foad 22. 23. end if 24. end fo β β 1.8 Agoithm 1. Psuedo code of the agoithm. 26

Key decisions such as 2-tuns of spheica spia and one econstuction in each of 1 steps ae made based on esuts fom initia expeiments. Undeined pats of the agoithm i be expained in the est of this chapte. Befoe 3D econstuction agoithm uns; the object has to be paced to the WCS oigin and motos have to be paced to thei statup position ( α and β 9, see Figue 4.3). Desciptions of undeined steps of the agoithm ae given in fooig sections. The agoithm oks as foos: Fist, the camea statup position is initiaized (ine 1 and ine 2. Then a ne image is captued fom the USB camea and undistoted (ine 3 and ine 4). Featue points of the image ae detected in ine 5 (see Section 5.1) and stoed in memoy (ine 6). Afte these initiaization steps, SSS pocesses stat in ine 7: A counte counts fom 1 to 4 hie the object otates 72 (4 steps of 1.8 tuns)(ines 18, 19). At each 8th step; camea otates one step (1.8 ) (ines 2,21,22,23). This eative movement makes the camea move in a spheica tajectoy (see Figue 4.4 and Figue 4.7). In each of 4 steps featue points ae tacked and stoed in the memoy (ines 8,9,1,11). At each 1th step of the counte; 3D econstuction is done (ine 13). Afte econstuction is done, scanning pocess is estated ith ceaed featue point coodinates (ines 14,15,16). Whie featues ae being einitiaized (ines 15 and 16) camea position is stationay. 5.1. Featue Point Detection A featue point is a point in an image that has a e-defined position and can be obusty detected (WEB_3 25). As in many compute vision appications, cones ae chosen to be used as featue points. In this study cvgoodfeatuestotack() function of OpenCV ibay is used. This agoithm detemines stong cones on images (WEB_2 24). A cone on images is seached in a sma egion of inteest using eigenvaues of the matix shon beo. A point on an image is identified as a cone hen eigenvaues coesponding to that point have high vaues. D D D 2 x x y C 2 D x D y D y (5.1) 27

hee D x and Dy ae the fist deivatives in x and y diections espectivey (WEB_2 24). Above matix is cacuated fo a pixe points on the image ith a sma egion of inteest. Eigenvaues ae found by soving a inea system coesponding to nxn matix A. AX λ X (5.2) The scaa λ is caed eigenvaue of A coesponding to eigenvecto X R n (WEB_4 26). Equation 5.2 can aso be itten as: ( A λ I) X (5.3) hee I is the identity matix. Equation 5.3 eads to det( A λ I) (5.4) When e sove equation 5.4 fo the 2x2 matix given in equation 5.1 then eigenvaues can be itten in cosed fom as: (WEB_2 24). 1 2 2 2 2 2 2 2 + ± + 2 λ ( ) 4( ( 1,2 D x D y D x D y D x D y Dx D y ) ) (5.5) 2 If λ 1 and λ 2 then thee is no featue in this pixe, if λ 1 and λ 2 is equa to some age positive vaue then an edge is found in that pixe, if both λ 1 and λ 2 ae age vaues, then a cone is found (WEB_5 25). These cones can be efined using some theshod vaue (t). If λ 1 > λ 2 >t hee t is some theshod, pixes that have thei eigenvaues ess than t ae eiminated and not consideed as cone points. The featue detection agoithm cvgoodfeatuestotack() that is buit in OpenCV oks as foo: The function fist cacuates the minima eigenvaue fo evey pixe of the souce image and then pefoms non-maxima suppession (ony oca maxima in 3x3 neighbohood emain). The next step is ejecting the cones ith the minima eigenvaue ess than quaityleve*<max_of_min_eigen_vas>. Finay, the 28

function ensues that a the cones found ae distanced enough fom one anothe by getting to stongest featues and checking that the distance beteen the points is satisfactoy. If not, the point is ejected (WEB_2 24). The function takes souce image, 2 image memoy fo eigenvaues cacuation pocess, a mutipie vaue (quaity eve) hich specifies minima accepted quaity of image cones, a imit vaue hich specifies minimum possibe Eucidean distance beteen etuned cones as input paametes. It etuns numbe of detected cones and image coodinates of detected cones. In expeiments.1 mutipie used fo quaity eve of cones and 1 pixe distance is used fo possibe distance beteen each cones detected. In this study e oked on to objects fo econstuction. Fist one is a e textued box ike object made of pape. This object has no concavities on its sufaces so it is eativey easy object to econstuct. Second object e used is piece of egg caton, eativey compex due to its concavities on sufaces and being not e textued. We put additiona paints on its suface to simpify featue detection and tacking. Images of oigina objects ae shon in Figue 5.2. (a) (b) Figue 5.2. Test objects used in this thesis fo 3D econstuction. (a) a box (b) a piece of egg caton. Figue 5.3 shos images of test objects afte featue point detection agoithm. Featue points ae epesented as bod cices. 29

(a) (b) Figue 5.3. Featue point detected images of test objects (a) box image ith 118 detected featue points (b) egg caton ith 97 detected featue points. 5.2. Tacking of Featue Points Tacking of featue points hie camea is moving on its spheica spia tajectoy aeviates the coespondence pobem. Since osing tacking of featue points i cause econstuction of the points in incoect positions, featue tacking is one of the most fundamenta steps in this study. In ode to tack a pixe in an image its veocity vectos have to be cacuated. This epesentation is caed veocity fied o optica fo fied of the image. Optica fo methods ty to cacuate the motion beteen to image fames that ae taken at times t and t+ t. Optica fo can be defined as an appaent motion of images bightness. Let I ( x, y, t) be the image bightness at time t of an image sequence. Main assumptions on cacuating optica fo is that: Bightness smoothy depends on a paticua image coodinates in geate pat of the image. Bightness of evey point of a moving o static object does not change in time (WEB_2 24). Optica fo fied cacuated fom image sequence of otating sphee is given beo. 3

Image #1 Image #2 Optica Fo Fied Figue 5.4. A otating sphee fom eft to ight and coesponding optica fo fied (Souce: WEB_6 26) If some point in an image is moving, ( x, y ). When Tayo seies fo bightness I ( x, y, t) is itten: t time ate the point dispacement is I I I I ( x + x, y + y, t + t) I ( x, y, t) + dx + dy + dt + c (5.6) x y t hee c epesents the highe ode tems hich ae sma enough to be ignoed. When above assumptions ae appied to equation 5.6, e have, I ( x + x, y + y, t + t) I ( x, y, t) (5.7) and I I I dx + dy + dt x y t (5.8) Dividing equation 5.8 by dt and e-aanging equation 5.8 such that dx u (5.9) dt dy v (5.1) dt 31

hee u and v ae speeds of point moving in x and y diections espectivey, gives the equation caed optica fo constaint equation: I I I u + v. (5.11) t x y I In equation 5.11 tem is about ho fast the intensity chages at a given pixe t I I in time. and ae about ho apidy intensity chages on going acoss the pictue. x y Diffeent soutions to equation 5.11 ae given in (Lucas and Kanade, 1981) and (Hon and Schunck,1981). We have used OpenCV impemetation of optica fo cacuation hich uses the Pyamidia Lucas&Kanade Technique. Reades ho ae inteested in detais about Lucas&Kanade technique shoud efe to (Lucas et a. 1981). Figue 5.5 shos a sequence of images hie 112 featue points ae being tacked. It can be easiy seen that the box otates in countecockise diection (aso can be imagined as camea otation aound the objectin cockise diection). Figue 5.6 shos a sequences of egg caton images ith 11 featue points being tacked. Image coodinates of a featue points ae ecoded in tacking pocess. (Coodinates (x,y) of an abitay featue point ae aso given in Figue 5.5.) 32

(a) Image #1 (148,17) (b) Image #2 (159,167) (c) Image #3 (171,166) Figue 5.5. Featue point tacking ove the box test object. 33

(a) Image #1 (b) Image #2 (c) Image #3 Figue 5.6. Featue point tacking ove the egg caton test object. 34

5.3. 3D Reconstuction of Tacked Featue Points Given camea paametes and featue point coespondences obtained by tacking in to vies, the goa i be to econstuct the 3D coodinates of the featue point. Lets ite camea s 3D positions in matix fomat assuming that cameas ae educed to one point (optica cente). Coodinates of the eft camea, x C y (5.12) z coodinates of the ight camea: x C y (5.13) z Since e kno the spheica coodinates of the camea defined by α, β and R, catesian camea coodinates (x,y,z) can be cacuated easiy by using equation 4.2. Whee α and β ae steppe moto s otation anges in cuent position of camea and R is the adius of the sphee shon in Figue 4.1. Image coodinates of any featue point (p) on eft image, i p (5.14) j coesponding coodinate of same featue point on ight image is: i p (5.15) j 35

hee i and j ae image coodinates in u and v diections espectivey. Fist e have to econstuct to ight ays (ines) L and L. A 3D ine equation in paametic fom can be itten as: L P + tv (5.16) hee P is stat point of ine, t is the paamete hose vaue is vaied to define points on the ine and V is a vecto defining the diection of the ine. When e ite the ines L and L stating fom camea s optica centes; L C + t V (5.17) and L C + t V (5.18) ines. In equation 5.17 and 5.18; t and t ae paametes that define points on both V and V ae vectos that define diections of both ines L and keep in mind that a vaiabes given above ae in WCS. L. We shoud Since the ony apioi infomation is on the image coodinates of featue points, V and V can be obtained by appying otation matix to CCS vectos V and V, hee V and pane (see Figue 5.7). V ae unit vectos fom optica cente to featue points on the image 36

37 Figue 5.7. The vecto defining the diection of econstuction ines. The 3D positions of the featue point p and the optica cente C ae: z y x p p p p o k p j p i p p z y x + + (5.19) z y x C C C C o k C j C i C C z y x + + (5.2) then econstuction ine diection vecto V is: C p V o z z y y x x C p C p C p V (5.21) C is the oigin of the camea coodinate system so k j i C + + and a images on image pane has 3d coodinates equa to camea s foca ength. If e put these togethe

38 f p p V y x (5.22) hee x p and y p ae featue point coodinates on image in u and v diections espectivey and f is foca ength (in tems of pixes) pe cacuated duing camea caibation. If pixes ae not squaes x p, y p and f have to be adjusted accodingy. The camea used in this study has squae pixes. If e ite featue point coodinates in vecto fomat using equation 5.14 and 5.15. f j i V o k f j j i i V + + (5.23) And f j i V o k f j j i i V + + (5.24) Equation 5.23 and 5.24 ae diection vectos in camea coodinate system. Befoe using them in equation 5.17 and 5.18 they have to be conveted to od coodinate sytem using tansfomation matix. If V and V ae coesponding unit WCS vectos of V and V then k j i V (5.25) k j i V (5.26)

39 hen e eite equation 5.17 and 5.18; + k j i t z y x L (5.27) + k j i t z y x L (5.28) In equation 5.27 and 5.28 e can see that paametic equations of to ines ae fuy knon. Next step is finding the intesection point o moe geneay finding the 3D point that is cosest to the to ines (see Figue 3.3). Paametes that makes to ines cosest to each othe ae given by 2 b ac cd be t c (5.29) 2 b ac bd ae t c (5.3) V V a V V b V V c S V d S V e hee C C S and ( ) denotes dot poducts of vectos. Appying equation 5.29 and 5.3 to equation 5.27 and 5.28 to 3D points i be acquied. The midde point

beteen these to points is etieved as the 3D econstucted point of the obseved featue point. Appying econstuction pocess to a featue points, a set of 3D points ae acquied. These points can be consideed as point couds ith no stuctua infomation. The next step is fitting a mathematica mode to the point couds (so-caed suface fitting ). Figue 5.8 shos econstucted point couds of test objects. (a) (b) Figue 5.8. Reconstucted 3D point couds of test objects. (a) box (b) egg caton. 4

Incoect econstucted points on the top of the box Figue 5.9. Reconstucted points of the box using non unifom ange. 41

CHAPTER 6 SURFACE FITTING The fina step of the 3D econstuction pocedue pesented in this thesis is fitting a paameteized 3D mode to the acquied point couds. Afte fitting such a mode to point couds, vaious high eve compute vision pocess can be appied to the 3D scene, such as textue mapping, 3D object ecognition etc. Thee ae many suface fitting techniques in iteatue (Siddiqui, M. and Scaoff, S. 22a), (Zicke et a. 24), (Yvat et a 25). The pesented method fo suface fitting is quite easy and fast, athough it cannot be appied to objects ith vaious topoogies. The technique is based on pojecting a econstucted points on to an initia meshed hemisphee that coves a points, and back pojecting the hemisphee s meshpoints toads the point couds. Choosing adius of hemisphee (R), equa to adius of spheica spia (see Chapte 4) is a good idea, since e kno that a econstucted points i be inside the hemisphee. When an (mxn) meshed, R adius hemisphee is initiaized a points ae pojected on to the hemisphee ecoding thei vecto magnitudes (Hee a econstucted points and mesh points ae epesented as vectos hee stat point of vectos ae WCS oigin and end points ae 3D point coodnates). Then ne coodinates of mesh points ae cacuated ith the hep of pojected vecto s magnitudes. The aveage of k-neaest vecto magnitudes ae set to be the magnitudes of mesh points. Finay initia vecto magnitude of mesh point is epaced ith that aveage magnitude keeping the diection of the initia vecto. Suface fitting agoithm is given beo (aso see Figue 6.1): 1. ceate a hemisphee ith mesh inteva (mxn) 2. poject a points on to hemisphee ecoding the diections and magnitudes. 3. fo a mesh points (mxn) a. find k-neaest pojected vectos on hemisphee to the mesh point (m,n) b. cacuate aveage magnitude of k-neaest vectos on the hemisphee c. set ne magnitude of mesh point as cacuated aveage magnitude d. cacuate ne Catesian coodinates of mesh point ith the hep of ne magnitude and keeping the initia diection. 42

Figue 6.1. A meshed hemisphee ith 2x15 Intevas. In the subject of suface fitting, e aso oked on synthetic data points. These point couds consist of 54 points. Athough the points ae ceated using mathematica equations, the eations beteen points ae assumed not to be knon in the suface fitting pocess. These synthetic data heped us to obseve the pefomance of suface fitting on point couds. Figue 6.2.a shos point couds coesponding to a synthetic object, hie Figue 6.2.b shos points pojected on the hemisphee and Figue 6.2.c shos back-pojected (fitted) hemisphee on the point couds. Duing suface fitting to synthetic data 1 neaest vectos ae used to cacuate the ne magnitude of mesh point. 43

(a) (b) (c) Figue 6.2. A synthetic data set ith 54 points (a), pojected data points on to hemisphee (b) and fitted suface on to data points ith mesh size (25x2) Figue 6.3 shos fitted suface to test object point couds. (a) Isometic vie of egg caton point couds couds (233 points) (b) fitted suface to egg caton point (25x2 size mesh) (c) Top vie of box point couds (2376 points) (d) top and side vie of fitted suface to box point couds (2x15 size mesh) Figue 6.3. Point couds ith coesponding sufaces. 44

Figue 6.4 shos a smooth suface mode of the egg caton. Fit function of The Mathematica softae is used to obtain the mode. The mathematica code is given beo hee EggC.txt is the text fie that contains econstucted 3D point coodinates. xyz ReadList["c:\EggC.txt", {Rea, Rea, Rea}]; poynomia Fatten[Tabe[x^i y^j, {i,, 4}, {j,, 4}]] f[x_, y_] Fit[xyz, poynomia, {x, y}] Pot3D[f[x, y], {x, -7, 7}, { y, -7, 7}, PotRange -> {-5, 5}, BoxRatios -> {6, 6, 5}, PotPoints -> 5, ViePoint -> {1, 2, -2}]; Figue 6.4. Egg caton suface. Fitted using the Mathematica softae. 45

CHAPTER 7 CONCLUSIONS In this thesis, an eectomechanica setup is deveoped to acquie images of test objects fom diffeent vie anges. The main diffeence beteen the deveoped setup and the one axis tun tabe based appications is that the deveoped setup has to otationa degees of feedom. We defined a spheica spia motion aound the test object hich heped us to get images not ony fom sides but aso fom top of the objects. Since steppe motos ae used fo otating the camea and objects, some mechanica vibations occu hich cause to ose tacking. These vibations can be educed using sevomotos instead of the steppe motos. Hoeve this soution i incease the cost and the compexity of the system. The inhomogenuity in ighting conditions is anothe impotant facto that degades system peomance. In the econstuction pocess featue tacking agoithm is used to aeviate the point coespondence pobem. Reconstuction is done by using steeo vision geomety hee one image is assumed to be eft image, and second image is assumed to be ight image. Featue points ae tacked beteen eft and ight image. Extinsic camea paametes of the vieing positions of eft and ight image ae cacuated by using steppe moto otation anges. Afte econstucting the points, ove spheica spia tajectoy an unoganized point couds ae acquied. Reconstuction time of an object depends on the tajectoy of the camea. Since mechanica vibations incease ith otation veocity of steppes causing tacking agoithm to ose featues, e added some deays beteen the steps of the agoithm. In this thesis e used a 2-tun spheica spia tajectoy move fo the camea hee econstuction time is about 2 seconds. Since mechanica moving time is much geate than the computationa time, computationa time fo econstuction is ignoed. To educe incoect econstucted points thee main pocess ae made. Fist a theshoding vaue added to the ine segment beteen cossing ines hich as descibed in Section 3.2. Second pocess is using unifom ange hie econstucting the points hich as descibed in Chapte 5. And ast pocess as to eiminate the featue points hich dispacements ae sma than some theshoding vaue hie 46

tacking. This pocess aeviate the incoect econstuctions of featue points that ae coected to the side of objects hie the object is otating. Suface fitting agoithm pesented in this thesis is quite easy and fast compaed to iteation based fitting techniques. Fo a point couds hich contains about 2 points, to fit 25x2 mesh size suface, compete fitting time is about 1 second on a PC ith a Pentium 4, 3GHz CPU, and 512 MB RAM. Athough this technique cannot be appied to a kind of objects such as objects ith hoes, the technique can be used ith appopiate objects due to its simpicitiy and speed. Textue mapping hich can be consideed as a possibe next step is not studied in this thesis. Athough the 3D econstuction and suface fitting esuts achieved in this thesis ae satisfactoy, the pefomance can be significanty impoved by hybid techniques (E.g. combining sihouette based econstuction techniques and featue tacking based econstuction techniques). A moe obust eectomechanica setup design (minimizing vibations), efinement in 3D econstuction, impoving suface modes and appication of textue mapping can be isted as the possibe eseach pobems that shoud be addessed in the futue. 47

BIBLIOGRAPHY Femont, V., and Cheai R. 24 Tuntabe-Based 3D Object Reconstuction ; IEEE Cybenetics and Inteigent Systems Confeence, Voume 2, Dec. 24, pp.1277-1282. Fosyth, D. A., and Ponce J. 23 Compute Vision: A Moden Appoach ; Pubished by Pentice Ha, Uppe Sadde Rive, Ne Jesey, Jan. 23. Hatey, R. and Zisseman A., 22 Mutipe Vie Geomety in Compute Vision ; Pubished by Cambidge Univesity Pess, 22. Lucas, B. D. and Kanade T. 1981 An iteative image egistation technique ith an appication to steeo vision ; Poc. 7 th Joint Conf. On Atificia Inteigence, Vancouve, BC, 1981, pp.674-679 Müayim, A. Y., Yımaz,U. And Ataay V. 23. Sihouette-based 3D Mode Reconstuction fom Mutipe Images ; IEEE Tansactions on Systems, Man and Cybenetics, Pat B, Voume 33, Issue 4, Aug. 23, pp.582-591. Nakbuakae, W., Pintaviooj C., Withayachumnanku W., Sangoasi M. And Taetuakan S. 25 3D Modeing fom Mutipe Pojections: Paae-Beam to Heica Cone-Beam Tajectoy, WSCG SHORT papes poceedings, Czech Repubic, Science Pess, Januay 31-Febuay 4, 25 Sainz, M., Baghezadeh, N., and Susin, A. 22. Caving 3D Modes Fom Uncaibated Vies ; Compute Gaphics and Imaging 22, Tack-Image Pocessing, 358-29 Sainz, M., Pajaoa, R., Mecade, A., and Susin, A. 24 A Simpe Appoach fo Point-Based Object Captuing and Rendeing ; IEEE Compute Gaphics and Appications, Voume 24, Issue 4, Aug.24, pp.24-33. Siddiqui, M. and Scaoff, S. 22a Suface Reconstuction fom Mutipe Vies using Rationa B-Spines ; IEEE 3D Data Pocessing Visuaization and Tansmission Symposium, 22, pp.372-378. Siddiqui, M. and Scaoff, S. 22b Suface Reconstuction fom Mutipe Vies Using Rationa B-Spines and Knot Insetion ; Poceedings Fist Intenationa Symposium on 3D Data Pocessing Visuaization and Tansmission, 22, pp.372-378. Wiiams, G. 23 Textued 3D Mode Reconstuction fom Caibated Images ; Univesity of Cambidge Dissetation 23 WEB_1, 26 Poefeys, M. - Comp29-89 Lectue Notes, Sping 23, 1/3/26. http://.cs.unc.edu/~mac/mvg/sides.htm 48

WEB_2, 24. Inte Open Souce Compute Vision Libay Manua, 2/12/24. http://.inte.com/eseach/m/eseach/opencv/ WEB_3, 25. Cone Detection,9/12/25, http://en.ikipedia.og/iki/cone_detection WEB_4, 26. Eigenvaue, 9/1/26, http://mathod.ofam.com/eigenvaue.htm WEB_5, 25. Cone Detection,9/12/25, http://mathod.ofam.com/cone_detection WEB_6, 26. Optica Fo Agoithm Evauation, 1/2/26, http://.cs.otago.ac.nz/eseach/vision/reseach/opticafo/opticafo Yvat, A., Hahmann, S., Bonneau, G.P. 25 Smooth Adaptive Fitting of 3D Modes Using Hieaachica Tiangua Spines ; Intenationa Confeence on Shape Modeing and Appications, SMI'5, 13-22 (25) MIT Boston, IEEE Compute Society Pess, 25. Yemez Y. and Schmitt F., 24 "3D econstuction of ea objects ith high esoution shape and textue,"; Image and Vision Computing, Vo. 22, pp. 1137-1153, 24. Zhang, Z. 1999 Fexibe Camea Caibation By Vieing a Pane Fom Unknon Oientations ; Seventh IEEE Intenationa Confeence on Compute Vision, Voume 1, Sept 1999, pp.666-673. Zhang, Z. 2 A Fexibe Ne Technique fo Camea Caibation ; IEEE Tansactions on Patten Anaysis and Machine Inteigence, Voume 22, Issue 11, Nov. 2, pp.133-1334 Zicke, M. and Gotsman, C., 24 Meshing Point Couds Using Spheica Paameteization ; Euogaphics Symposium on Point-Based Gaphics, Zuich, June 24. 49

APPENDIX A COMPUTER SOFTWARE In this thesis, Micosoft Visua C++ 6. and OpenCV ibaay is used to deveop the equied agoithm. Matab is aso used to cacuate the camea intinsic matix. The deveoped 3D econstuction system is consist of to individua pats. Fist one is econstuction and second one is fitting. The pogams that incudes these agoitms i be biefy descibed beo. These compute pogams ae given in a CD. This CD aso contains a README text fie that descibes the compute pogams. mscthesis.cpp: This is the souce code that contains econstuction agoithm. The code oiginay modified fom kdemo.c hich is the demo softae of Lucas&Kanade tacking agoithm suppied by OpenCV Libay. This souce code gets images fom USB camea, contos the eectomechanica component via paae pot and ites the econstuction esuts into text fies. mschesis.cpp. otate.h :This souce code cacuates the otation matix needed by ine3d.h: This souce code cacuates the cosest points of to 3D ines. The code aso contains some gemetica cacuations such as intesection of ine and pane, pane point distance etc. SufaceFit.cpp: This souce code takes a text fie that contains 3D point coodinates as an input. The text fie is ceated by mscthesis.cpp. Then this agoithm cacuates a suface appoximation to that points. Mesh point coodinates ae stoed in a text fie as a esut of fitting pocess. 5