COMPUINGPARAMERICGEONDESCRIPIONS OF3DMULI-PAROBJECS DepartmentofElectricalEngineering KenongWu McGillUniversity April996 AhesissubmittedtotheFacultyofGraduateStudiesandResearch inpartialfullmentoftherequirementsofthedegreeof DoctorofPhilosophy ckenongwu,996
Abstract ABSRAC single-andmulti-viewrangedataispresented.hisresearchismotivatedbybothatheory ofhumanimageunderstanding(recognition-by-components)andtheneedforqualitative mentȯbjectdescriptionsareobtainedintwoconsecutivesteps:()objectsegmentationinto recognitionbyanautonomousrobotinorderforittoecientlyinteractwithitsenviron- Anewapproachforcomputingqualitativepart-baseddescriptionsof3Dobjectsfrom partsand(2)partmodelidentication.segmentationisachievedbyrstcomputingthe surface.healgorithmthendetectstheobjectpartboundarieswherethethechargedensity partmodels,whichindicatebothqualitativeshapeandquantitativeattributeinformation. indicationofthegrossandneobjectstructures.parametricgeonsareintroducedasthe simulatedelectricalchargedensitydistributiononatessellatedtriangularmeshoftheobject thebestmodelbasedontheminimumttingresidual.anewobjectivefunctionused formodelrecoveryisoptimisedbyaglobaloptimisationtechnique(veryfastsimulated Modelrecoveryisachievedbyttingallparametricgeonstoapartandthenselecting achievesalocalminimum.hechargedensitydistributioncansimultaneouslyprovidean Re-Annealing). aphysicalanalogytothewellknowntransversalityprinciple,partsegmentationdoesnot putelocalsurfacefeatures.heformulationforparametricgeonsprovidesaglobalshape requireanassumptionofsurfacesmoothnessorthechoiceofaparticularscaletocom- constraint,whichensuresreliablepartmodelrecoveryevenwhenthepartshapeisnot anexactinstanceofaparametricgeon.bydirectlycomparingapartwithallcandidate headvantagesofthisapproacharedemonstratedthroughexperimentation.byusing computedpart-baseddescriptionsarewellsuitedfortheobjectrecognitiontaskcarriedout byanautonomousrobot. models,thisapproachexplicitlyveriestheshapeoftheresultantpartdescriptions.he ii
Resume RESUME Cetravailpresenteuneapprochepourlecalculqualitatifdeladescriptionenpartiesd'objets 3D,percusapartird'uneoudeplusieursvuesdedistance.Cetterechercheestmotivee alafoisparlatheoriedecomprehensiondesimagesdel'^etrehumain(reconnaissancepar enpartieset(2)l'identicationdecesparties.lasegmentationestaccomplieencalculant composantes)etlebesoind'unereconnaissancequalitativepermettantaunrobotautonome d'interagirecacementavecsonenvironnement. dechargepeutegalementfourniruneindicationsurlesstructuresnesetgrossieresde l'objet,dontladensitedechargeatteintunminimumlocal.ladistributiondedensite toutd'abordlasimulationdedistributiondedensitedechargesurtesselationtriangulairedelasurfacedel'objet.l'algorithmedetectealorslesfrontieresdescomposantesde Lesdescriptionsd'objetssontobtenuesendeuxetapes:()lasegmentationd'objet l'objet.lesgeonsparametriquesserontlemodeledespartiesd'objet,etindiquerontala foisuneinformationqualitativedeformeetuneinformationquantitative.laselectiondu modeled'unepartieestobtenueparlamiseencorrespondancedesmesuresetdesgeons parametriquesetenselectionnantlemodelequiobtientl'erreurresiduelleminimale.une nouvellefonctionobjectifutilisepourlareconnaissancedemodeleestoptimiseeparune techniqued'optimisationglobale(recuitsimuletresrapide). necessitepasd'hypothesedesurfacelisseoulechoixd'uneechelleparticulierepourcalculerlescaracteristiqueslocalesdelasurface.laformulationdesgeonsparametriques partiem^emelorsquelaformedecettedernieren'estpasuneinstanceexacted'ungeon fournieunecontrainteglobaledeformequiassureuneidenticationabledumodeled'une Lesavantagesdecetteapprochesontdemontresparl'experimentation.Enutilisant l'analogiephysiquebienconnueduprincipedetransversalite,lasegmentationenpartiene scriptionenpartiesainsicalculeess'averentbienadapteespourlest^achesdereconnaissance cetteapprocheverieexplicitementlaformedesdescriptionsenpartieresultantes.lesde- parametrique.parunecomparaisondirected'unepartieavectouslesmodelescandidats, d'objetrequisespartoutrobotautonome. iii
hisdissertationisdedicatedtomyparents FengZhi-ZhongandWuDe-Qing andtomywife Jing iv
Acknowledgement ACKNOWLEDGEMEN MartinD.Levineforhispersistentguidance,supportandencouragement.Hisinsightful searchandthesiswriting.firstandforemost,iwouldliketothankmysupervisorprofessor Manypeoplehaveprovidedguidance,assistanceandencouragementduringmystudy,re- manycommentsonglobaloptimisationtechniques.iwouldalsoliketothankprofessor JonathanWebboftheDepartmentofElectricalEngineeringatMcGillUniversity,who leadmeintotheeldofelectrostaticsandprovidedvaluableadviceonchargedensity Re-Annealingalgorithm.Heprovidedthesoftware,guidanceforusinghissoftwareand adviceandvaluablecriticismmadethecompletionofthisworkpossible. computations. IwouldliketothankProfessorLesterIngber,theoriginatoroftheVeryFastSimulated University.hestimulatingresearchenvironmentandfriendlyatmospheremademylatest studentlifepleasant.iamthankfultoprofessorfrankf.ferrieandhisgraduatestudents fordeveloping3dvisionsoftwaretools,whichmademyprogrammingworkmucheasier.i wishtothankduncanbaird,donbui,hierrybaron,gerardblais,marcbolduc,michael Kelly,MichaelLanger,KaleemSiddiqi,GilbertSoucy,PeterWhaiteandJohnZelekfortheir ItwasapleasuretobeamemberoftheCentreforIntelligentMachinesatMcGill love,understanding,encouragementandpatience. portedmeinmypursuitofmycareergoals.manythanksareduetomywife,jing,forher technicalassistanceandmanydiscussions.mythanksarealsoduetonormaprocyshyn andornellacavalierefortheiradministrationassistance. Iwouldliketoexpressmydeepestgratitudetomyparents,whoencouragedandsuptainedfromthePRIPLabattheMichiganStateUniversityunderthedirectionofProfessor triangularmeshesfortherangenderdata.certainrangeimagesusedinthisthesiswereob- AnilK.Jain. ofpennsylvania),anddr.marcsoucy(innovmetricsoftwareinc.)forkindlyproviding IwishtothankDouglasDeCarlo,ProfessorDemitriMetaxas(GRASPLab,University CouncilofCanadaStrategicGrantandanFCARGrantfromtheProvinceofQuebec.v hisresearchwaspartiallysupportedbyanaturalsciencesandengineeringresearch
ABLEOFCONENS ABLEOFCONENS Abstract:::::::::::::::::::::::::::::::::::::::::::ii Resume:::::::::::::::::::::::::::::::::::::::::::iii LISOFABLES:::::::::::::::::::::::::::::::::::::xiii LISOFFIGURES::::::::::::::::::::::::::::::::::::x Acknowledgement:::::::::::::::::::::::::::::::::::::v ::::::::::::::::::::::::::::::::::::::::::::::::iv CHAPER.Introduction::::::::::::::::::::::::::::::: CHAPER2.RelatedWork:::::::::::::::::::::::::::::::4 2.StatementoftheProblem:::::::::::::::::::::::::::::4.Background:::::::::::::::::::::::::::::::::::::3 3.hesisOverview:::::::::::::::::::::::::::::::::::9 4.ClaimsofOriginality::::::::::::::::::::::::::::::::.PartSegmentation:::::::::::::::::::::::::::::::::4 2.PartModels:::::::::::::::::::::::::::::::::::::2..Shape-basedApproaches:::::::::::::::::::::::::::5.2.Boundary-basedApproaches:::::::::::::::::::::::::7 CHAPER3.MotivationandMethodology::::::::::::::::::::::28 3.ModelRecovery:::::::::::::::::::::::::::::::::::24 4.ChapterSummary:::::::::::::::::::::::::::::::::27 2..QualitativeModels:::::::::::::::::::::::::::::::2.OverviewofRBCheory:::::::::::::::::::::::::::::28 2.2.QuantitativeModels::::::::::::::::::::::::::::::22 vi
3.UnambiguousDenition::::::::::::::::::::::::::::::30 4.ShapeApproximation::::::::::::::::::::::::::::::::30 2.FromLineDrawingstoRangeData::::::::::::::::::::::::29 5.AmountofInformation:::::::::::::::::::::::::::::::32 ABLEOFCONENS 8.GeneralFramework:::::::::::::::::::::::::::::::::34 7.GeneralAssumptions::::::::::::::::::::::::::::::::33 9.ComparisonwithPreviousWork:::::::::::::::::::::::::35 6.PartSegmentation:::::::::::::::::::::::::::::::::32 0.ChapterSummary:::::::::::::::::::::::::::::::::4 9.3.ParametricGeonModels:::::::::::::::::::::::::::38 9.4.PartModelRecoveryStrategy::::::::::::::::::::::::40 9.2.QualitativeShapeApproximation::::::::::::::::::::::37 9..PhysicalModel:::::::::::::::::::::::::::::::::35 CHAPER4.ObjectSegmentationintoParts::::::::::::::::::::42 3.Computation::::::::::::::::::::::::::::::::::::46 4.FiniteElementSolution::::::::::::::::::::::::::::::48 5.CharacteristicsofChargeDensityDistribution::::::::::::::::::5 2.AssumptionsaboutObjectShapes::::::::::::::::::::::::46 6.Surfaceriangulation::::::::::::::::::::::::::::::::53.Physics::::::::::::::::::::::::::::::::::::::::42 7.PartDecomposition:::::::::::::::::::::::::::::::::57 6.2.ConstructingariangularMeshforMultiviewData::::::::::::55 6..ConstructingariangularMeshforSingle-ViewData:::::::::::54 CHAPER5.PartIdentication::::::::::::::::::::::::::::6.ParametricGeons::::::::::::::::::::::::::::::::::6 8.ChapterSummary:::::::::::::::::::::::::::::::::59 7..DirectConnectionGraph:::::::::::::::::::::::::::57..Shapeypes::::::::::::::::::::::::::::::::::6 7.2.FindingParts::::::::::::::::::::::::::::::::::58 2.ComparisonwithOriginalGeons:::::::::::::::::::::::::68 3.heObjectiveFunction::::::::::::::::::::::::::::::69.2.Formulation:::::::::::::::::::::::::::::::::::63 3..heDistanceMeasure:::::::::::::::::::::::::::::70 vii
4.MinimisingtheObjectiveFunction::::::::::::::::::::::::74 3.2.heNormalMeasure:::::::::::::::::::::::::::::73 4..Optimisationechnique::::::::::::::::::::::::::::74 3.3.BiasingtheObjectiveFunctionwithDierentNorms:::::::::::74 ABLEOFCONENS CHAPER6.Experiments:::::::::::::::::::::::::::::::80 6.ChapterSummary:::::::::::::::::::::::::::::::::78 5.Discussion::::::::::::::::::::::::::::::::::::::77 4.2.DeterminingtheParameterSpace::::::::::::::::::::::76.DataAcquisition::::::::::::::::::::::::::::::::::80 4.3.StoppingConditions::::::::::::::::::::::::::::::77 4.ObjectDecomposition:::::::::::::::::::::::::::::::92 2.ParameterSpecications::::::::::::::::::::::::::::::8 5.PartIdentication:::::::::::::::::::::::::::::::::96 3.ChargeDensityDistribution::::::::::::::::::::::::::::82 5..UsingRangeDataofGeon-likeObjects:::::::::::::::::::97 5.2.UsingRangeDataofImperfectGeon-likeObjects:::::::::::::98 3.2.3DCase:::::::::::::::::::::::::::::::::::::90 3..2DCase:::::::::::::::::::::::::::::::::::::83 5.3.ComparingDierentObjectiveFunctions::::::::::::::::::00 CHAPER7.CONCLUSIONS:::::::::::::::::::::::::::::08 6.ChapterSummary:::::::::::::::::::::::::::::::::05 5.4.ComparingSingle-viewandMultiviewData:::::::::::::::::03 5.5.ComparingPerfectandImperfectGeon-likeObjects::::::::::::03 2.hesisContributions::::::::::::::::::::::::::::::::0.hesisSummary::::::::::::::::::::::::::::::::::08 5.6.UsingMulti-partObjects:::::::::::::::::::::::::::05 APPENDIXA.IntegralEvaluation:::::::::::::::::::::::::::28 REFERENCES:::::::::::::::::::::::::::::::::::::::5 4.FutureWork:::::::::::::::::::::::::::::::::::::2 3.Limitations::::::::::::::::::::::::::::::::::::: APPENDIXB.MultiviewIntegration:::::::::::::::::::::::::3 viii
7 9 0 6 3 4 8 2 5 2 7 9 0 6.DCGCONSRUCION 3 2 2 4 5 8 (a) (b) 2 3 4 5 6 3 2 3 4 5 6 3 4 2 (c) (d) 3 4 2 3 4 2 2 3 2 3 5 2 3 2 2 3 2 2 4 4 5 5 2 6 6 (e) (f) 3 4 2 3 4 2 2 3 5 6 3 2 3 5 6 3 describedasfollows: FigureD..ConstructionoftheDirectConnectionGraph(DCG).(a)Atriangularmesh.(b) herststep.columnshowsthelistoftrianglesinthemesh.column2tocolumn4show 3 2 2 3 2 0 3 AlgorithmD.. Foranexample,inthecaseof;D[;5]=2.healgorithmforconstructingthisarrayis thersttothethirdneighbour,respectively,ofthetrianglegivenintherstcolumn.helast columnshowsthenumberofdirectneighbourssofar.(c)to(f)indicatesuccessivestepsofdcg 4 4 8 2 5 2 5 2 8 2 3 (i)loadalltrianglesinthemeshintod[i;]andsetd[i;5]=0. construction.(7)showsthenalresultforto6. 6 2 6 2 9 3 4 (g) (h)
(ii)startwith,foreachi, (a)searchdownwardintherstcolumnforadirectneighbourofi,basedon Denition4.3.Whenadirectneighbouriofjisfound, (i)putjintheneighbourlistofi, 2.FINDINGPARBOUNDARIES (iii)endforeachi. SincetheDCGprovidesanexplicitrelationshipbetweenindividualtrianglesonthe (b)fromj,repeatthesearchprocedurein(a)untilnisreached. (iii)increasebothd(i;5)andd(j;5)by. (ii)putiintheneighbourlistofj, Itpermitsthetracingofthepartboundariesonthetriangularmeshwithoutemployinga surfaceoftheobject,itservesasaconvenientcoordinatesystemovertheobjectsurface. describingtheobjectandincreasesthecomputationalspeed. 2.FindingPartBoundaries voxel-basedcoordinatesystem.hissignicantlyreducestherequiredmemoryspacefor bydeepsurfaceconcavities.foracompleteobject,thepartboundaryisaclosedsurface. intersectingsurfacesareofdierentorientationsatallpointswherethesurfacesintersect transversallywithprobabilityone[].hismeansthatthetangentplanestothetwo (seefigure2.5).followingthis,wehaveassumedthatapartboundaryisexplicitlydened hisensuresthatthedecompositionalgorithmwillbeabletosegmentapartfromtherest hetransversalityprinciplestatesthatwhentwosurfacesintersect,theyintersect procedure.sincethepartboundaryislocatedatlocalchargedensityminima,itcanbe tracedalongthe`valley'ofthechargedensitydistribution.wenotethatthetracingalgorithmappliedtoaclosedmeshisslightlydierentfromthatappliedtoanopenmesh.he oftheobject.heassumptionalsoprovidesastoppingcriterionfortheboundarytracing describethealgorithmfortracingtheclosedtriangularmeshes. onlytwodirectneighbours.sincetwoalgorithmsareverysimilar,inthefollowing,weonly oftheopenmesh,itstops.hetriangleonthemeshboundaryisdenedastheonewith henewcriterionisthatwhenthetracingalgorithmreachesatriangleontheboundary anopenmeshisnotclosed,thestopcriterionfortracingontheclosedmeshismodied. latterisconstructedforrepresentingsingle-viewrangedata.sincethepartboundaryon fortracingeachboundary.astartingtrianglemustsatisfythefollowingconditions: (i)itmustbeaconcaveextremum;thatis,itschargedensitymustbealocalminimum. healgorithmexaminesthechargedensityonalltrianglestondastartingtriangle 42
(iii)itanditsneighboursmustnothavebeenvisitedbefore.hisensuresthatthesame (ii)itmustbelocatedatadeepconcavity.husthechargedensityonthetrianglemust belowerthanapreselectedthreshold2. boundarywillnotbetracedagain. 2.FINDINGPARBOUNDARIES thedcg.heprocesscontinuesuntilitreturnstothestartingtriangle.asaresultofthe assumptionstatedatthebeginningofthissection,thismeansthatalltrianglesonthispart boundaryhavebeenvisited.nextthealgorithmndsanewstartingtriangleandtraces marked.hemarkedoneswillnotbecheckedagainandeventuallywillbedeletedfrom chargedensity.duringthetracingprocedure,alltrianglesdetectedontheboundaryare Beginningatthestartingtriangle,thealgorithmproceedstotheneighbourwiththelowest deleted.hustheoriginaldcgisnowdividedintoasetofdisconnectedsubgraphs,as showninfigure4.0(c).physicallytheobjecthasbeenbrokenintoparts.eachobject partcanbeobtainedbyapplyingacomponentlabellingalgorithmtoasubgraphofthe densityatastartingtriangleishigherthanthepreselectedthreshold.afteralltriangles onpartboundarieshavebeenfound,thenodesofthedcgrepresentingthesetrianglesare anotherboundary.itrepeatsthesametracingprocedure,andnallystopswhenthecharge DCG.heresultantpartisreadyforpartmodelrecovery. usedinthealgorithmaredenedasfollows: Inthefollowing,wedescribetheboundarytracingalgorithmindetail.hevariables Bi:Asetoftrianglesthatbelongtoboundariesofparts,whereiisthenumberof t:thethresholdforastartingpatch (j):thechargedensityofthejthpatch min:thelowestchargedensityoveralltriangularpatches,i.e.,triangles. NewPatch:Aagindicatingwhetheranewpatchonthepartboundaryisfound start:asetoftriangleswithwhichtheboundarytracingprocedurestarts. boundaries. Duringthetracing,welabelallpatchesasfollows: 2histhresholddetermineswhenanobjectshouldnotbedecomposedanyfurther.Ifthechargedensityata MiniCharge:thechargedensityforstart. PminiCharge:theminimumchargedensityofadirectneighbourpatch. P0:thepatchesbelongtoBi (NewPatch=)ornot(NewPatch=0). lowestchargedensityontheobjectsurfaceasthethreshold. it.obviously,thehigherthethreshold,themoresegmentedpartswillbefound.currentlywechoose20%ofthe ofthethresholddependsonaprioriknowledgeofthesurfaceconcavityandthereisnouniversalruleforselecting startingtriangleisgreaterthanthisthreshold,weassumethatallboundarypointshavebeenfound.heselection 43
heboundarytracingalgorithmisdescribedasfollows: P:thepatcheshavebeenvisited P2:theneighboursofstartwhichhavebeenvisited P3:thepatchesthathavenotbeenvisited 3.CLASSIFYINGPACHESINOPARS AlgorithmD.2. (ii)loopforeachpartboundarycontouri (i)markalltriangularpatchesasp3. (b)loopforj (a)initiallysetminicharge=t. (d)ifminicharge<t (c)endofloop (ii)if(j)<minicharge,assign(j)tominichargeandmarkjas (i)findapatchjatalocalchargedensityminimum. (ii)loopfork,eachneighbourofstart (i)setallneighboursofstarttop2. start. (iii)endofloopfork. (iv)ifnewpatch=,marknasp. (v)loopfork (A)IfNBkisaP3,assignthechargedensityofNBktoPminiCahrge, (A)If(k)isaP3,markitasP. setnewpatch=,markthispatchasnandescapetheloopfor (B)If(k)<PminiCharge,assign(k)toPminiCharge. (viii)if(k)isaneighbourofstartoraneighbourofbi,stoptracingthe (vii)mark(k)asp0. (vi)endofloopfork 3.ClassifyingPatchesintoParts (iii)endofloopfori Inthissection,wedescribeacomponentlabellingalgorithmforatriangularmesh.We contouri. wouldliketoobtainalltriangularpatchesinaparticularsubgraphofthedcg,which 44
3.CLASSIFYINGPACHESINOPARS neighbors patch 7 9 0 6 3 4 8 2 5 2 (a) FigureD.2.Componentlabelling.(a)aDCGcontainingapartboundary,asindicatedbythe shadedpatches.(b)adcgarray.(c)theupdatedpararrayfromlefttoright.heindexes 2 3 4 5 6 7 8 9 0 2 3 3 2 2 9 4 6 3 6 5 4 5 2 8 8 9 0 5 7 7 2 6 0 2 (b) 3 3 3 4 4 threeexplicitlyspeciedneighbours.anexampleofthisalgorithmisshowninfigured.2. representsoneobjectpart.heexistingcomponentlabellingalgorithms[07,06]are mainlyfor2dbinaryimages,inwhicheachpixelhasfourdirectneighbours.hedierence betweena2dimageandatriangularmeshisthatthelatterisdescribedasadarraywith intherightmostarrayindicatepatchesthatbelongtothesamepart. 4 AlgorithmD.3. healgorithmndseachpartinsequenceandisillustratedasfollows: 7 (c) (i)loopforeachobjectparti 45
(b)startingwiththersttriangleinthedcgarray,ndtherstuncheckedtrianglewhichisnotonapartboundary.addittoparandmarkitinthe partandinitialiseacounterforthenumberoftrianglesoneachpart. 3.CLASSIFYINGPACHESINOPARS (a)openatemporaryspace,calledpar,tostorethetriangleindexfortheith (d)inthedcgarray,movethepointerdownoneelement.repeatstep(c)until (c)inthedcgarray,addtheneighboursofthistriangle,which()arenotona DCGarrayasacheckedtriangle.SpecifyapointerintheDCGarray,pointing (e)markthetrianglesinparasbeingonthesameparti.hustriangles nomoretrianglescanbeeithercheckedorareonboundaries. tothistriangle. partboundaryand(2)havenotalreadybeeninpar,intopar. belongtothesameobjectpart.hesetrianglelistsareusedforpartmodelidentication. (iii)repeatfromstep(a)untilalltriangleshavebeenchecked. (ii)endofloopfori heresultofthisalgorithmisseverallistsoftriangles.eachlistcontainstriangleswhich belongingtopartihavebeenfound. 46
DocumentLog: ypesetbyams-laex 24April996 ManuscriptVersion0 KenongWu CentreforIntelligentMachines,McGillUniversity,Montreal,Quebec,Canada E-mailaddress:wu@cim.mcgill.ca ypesetbyams-laex