Generictoolsarenotcurrentlyavailablewithwhichtoaccuratelydeneimagecontent.The

Transcription

1 AFractal-BasedClusteringApproachinLargeVisual StateUniversityofNewYorkatBualo DepartmentofComputerScience AidongZhangandBiaoCheng DatabaseSystems DepartmentofElectricalandComputerEngineering StateUniversityofNewYorkatBualo Bualo,NY14260 Bualo,NY14260 RajAcharya assistecientaccesstoimagecontent.withthelargevolumeofvisualdatastoredinavisual database,imageclassicationisacriticalsteptoachieveecientindexingandretrieval.in fromimagedataintheirpixelformat.thesefeaturesmustthenbeclassiedandindexedto visualdataonthebasisofcontent.inthisprocess,signicantfeaturesmustrstbeextracted Largevisualdatabasesystemsrequireeectiveandecientwaysofindexingandaccessing Abstract thispaper,weinvestigateaneectiveapproachtotheclusteringofimagedatabasedonthe proposedforthemeasurementoftheextentofsimilarityamongimages.byexperimentingon todeterminethedegreeoftheirsimilarity.imagesinavisualdatabasecanbecategorizedin compressiontechnique.ajointfractalcodingtechnique,applicabletopairsofimages,isused clustersonthebasisoftheirsimilaritytoasetoficonicimages.classicationmetricsare techniqueoffractalimagecoding,amethodrstintroducedinconjunctionwithfractalimage 1Introduction Whilelargevisualinformationsystemsrequireeectiveandecientmeansofindexingandaccessingimagesbasedontheircontent,researchtowardachievingthisgoalisstillinapreliminarystage. theproposedclusteringtechniquetovariousvisualdatabaseapplications. alargesetoftextureandnaturalimages,wedemonstratetheapplicabilityofthesemetricsand semantics.ecientdataaccesscanbesupportedbyb-treesandotherdatastructures.however, Methodsforindexingandaccessingalphanumericdataintraditionaldatabasesare,incontrast, wellunderstood.inthesedatabases,categorizationofdataisstraightforwardonthebasisoftheir 1

2 suchtraditionalapproachestoindexingmaynotbeappropriateinthecontextofcontent-based visualdataretrieval[acf+93,cyda88,tpf+91,rs91,gs,hlhc92,bpj93,wn94].inthis context,visualqueriesareposedviavisualorpictorialexamples.atypicalvisualquerymight suchasatumor,inadiagnosticimage. image.suchaquerycould,forexample,beusedinamedicalimagesettingtondananomaly, entailthelocationofallimagesinadatabasethatcontainasubimagesimilartoagivenquery acriticalroleonsupportingecientindexing.toecientlyhandleretrievalonalargevolume interpretation.thus,theabilitytoautomaticallyclusterimagesonthebasisofcontentassumes detectingandclassifyingimagesisbothhighlyinecientandpronetoerrorsarisingfromsubjective culties.giventhelargevolumeofimagedatacollectedinavisualdatabase,amanualapproachto automaticcategorizationandretrievalofimagesonthebasisofcontentthusposesignicantdi- Generictoolsarenotcurrentlyavailablewithwhichtoaccuratelydeneimagecontent.The ofimages,theseimagesmustberstclassiedintovariouscategoriesbeforeindexingonimage contentfeaturesareperformed.theclusteringofimagesintocategoriesonthebasisofcontentis canbeusedtosupportcontent-basedimageretrieval.recentdevelopmentsintheeldofimagedata coarseness,contrast,anddirectionality[tmy78,nbe+93,wn94].thesetexturecharacteristics featuresofimagesfromtheirmathematicalrepresentationsusedinvariousimagecompression techniques.textureisanimportantunderlyingprimitiveinhumanvisualperception,including thusaprerequisitetotheeectiveandecientexecutionofvisualqueries. transformationandcompressionprovideaninterestingapproachtodescribingthecontentofimage Anapproachwhichhasdrawnmuchrecentattentioninvolvestheextractionofthetexture texture.imagedatatransformationandcompressioneliminatedataredundancybydecorrelating thepixelvaluesofimagedata.theresultingtransformeddatarepresentsthesalientcharacteristics oftheimage,andthesecanbeusedtoperformimageretrieval.amongtheimagecompressionmethods,jpeg[wal91],wavelettransformation[hjs94],andfractalcoding[jac93,slo94,bs88,fl92] haverecentlydrawnmuchattention.ithasbeendemonstratedthatboththediscretecosine Transform(DCT)usedinJPEGandwavelettransformationcanbeusedfortextureclassicationanddiscrimination[SC94a,SC94b].Fractalimagecompression,acompactedmathematical representationofimagedata,alsooersapromisingapproachtothedescriptionofthetexture featuresofimagedata.initialresearchinthisdirection[slo94,zca95a,zca95b,zcam96]has demonstratedthatfractalcodingofimagedatacanbeusedtorecognizeusefultexturefeatures whichcanassistcontent-basedimageretrieval. database.usingthisclusteringtechnique,aclusteringtoolcanbebuilttoassisttheautomatic categorizationofimagesinalargevisualdatabaseonthebasisoftheirsimilaritytoasetoficonic images.indexingstrategiesshouldthenbedesignedonimagefeaturevectorsineachcategory. Inthispaper,weinvestigateatechniqueforecientclusteringofimagesinalargevisual 2

3 Throughthisclusteringstrategy,ecientretrievalofvisualdatabasedontheircontentcanbe sentation.inthisapproach,thetechniqueoffractalcodingisappliedtoimageclusteringthrough supported. areproposedforthemeasurementoftheextentofsimilarityamongimages.thisprocessoperates itsextensiontojointfractalcodingbetweenimages.thisapproach,applicabletopairsofimages, canbeusedtodeterminethedegreeoftheirsimilarity.imagesinavisualdatabasecanbecategorizedinclustersonthebasisoftheirsimilaritytoasetoficonicimages.classicationmetrics Theproposedclusteringapproachisdevelopedbasedontheextensionoffractalimagerepre- applications.agenericclusteringtoolhasbeendevelopedwhichcanbeusedinvariousvisual databaseapplications. automatically,withnohumanassistancerequiredtodeterminethepointsofsimilaritybetween offractalcodingforimagecompressionanditspotentialonimageclustering.insection3,we images.theproposedclusteringtechniqueiswell-suitedtoincorporationintomanydigitallibrary proposethejointfractalcodingtechniquewhichcanbeusedtosupportimageclustering.section 4introducessimilaritymeasuringmetricstodeterminethedegreeofsimilaritybetweenimages. Experimentalresultsofclusteringonnaturalimageswillalsobepresented.InSection5,we discusstherelationshipbetweenclusteringandindexing.otherpotentialclusteringapproachesare Therestofthepaperisorganizedasfollows.Section2discussesthemaincharacteristics alsoconsidered.concludingremarksanddirectionsforfutureresearchareoeredinsection6. basedimageclustering. 2Background 2.1Fractal-BasedImageCompression andthendiscussthecentralroletobeplayedbyfractal-basedsimilaritycomparisonincontent- Inthissection,wewillrstprovideanintroductiontothetechniqueoffractalimagecompression, eachsuchimagerange,thefractalencoderndsalargerblockwithinthesameimage(termed tions.theformalmathematicaldescriptionofifscanbefoundin[fis95].jacquin[jac92,jac93] proposedafullyautomatedalgorithmforfractalimagecompression.informally,fractalimage compressionpartitionsanimageintoacollectionofnon-overlappingregions,termedranges.for Fractalimagecompressionisbasedonthemathematicalresultsofiteratedfunctionsystems(IFS) thedomainblock,d)suchthatatransformationw(d)ofthisblockisthebestapproximationof [SH94].Barnsleyetal.[BS88]rstrecognizedthepotentialofIFSforcomputergraphicsapplica- therangeblock,accordingtoanappropriatecriterionofsimilarity.thefractalcodefortherange consistsofthegeometricalpositionsoftherangeanddomainaswellasthetransformation.while 3

4 fractalcode. thepixeldata,ahighcompressionratiocanbeachieved[fis95].principally,thisapproachassumes thatimageredundancycanbecapturedandexploitedthroughpiecewiseself-similaritytransformations[jac93].theoriginalimagecanbeapproximatedfromanitenumberofiterationsofits thepixeldatacontainedintherangeandinthedomainareusedtodeterminethefractalcode,they arenotpartofthecodeitself.asthetransformationsaremuchmorecompactlyrepresentedthan andb1;:::;bn,respectively.weusetherms(rootmeansquare)metrictocomparethedistance pixelsastherange.letareducedblockdjandtherangericontainnpixelintensitiesa1;:::;an thattheimagepartitionisgeneratedbyaquadtreescheme[fis95].inthisscheme,imagesare rangesizeandthenaveragegroupsof22pixelstogetareducedblockwiththesamenumberof partitionedintosquarerangeblocks.foreachrangeblock,weselectadomainblockwithtwicethe Wenowprovideimplementationdetailstoexplainthiscompressiontechnique.Weassume betweendjandri.thermsdistancedistibetweendjandriiscomputedasfollows: formedakvalueshavetheleastsquareddistancefromthebkvalues.detailsonthecalculationsof (k=1;:::;n).thiswillprovideuscontrastandbrightnesssettingsthatmaketheanelytrans- siandoicanbefoundin[fis95].outofthepossibledomainblocks,adomainblockwithadistance wheresiandoiarevalueswhichminimizedistidenedinformula(1)forthegivenak;bk Disti=sPnk=1(siak+oi?bk)2 n therearetwospatialdimensionsandthegreyleveladdsathirddimension.thus,thetransformationwifortherangeriisacombinationofageometricaltransformationaswellasluminance belowthetolerancelevel1(computedbyformula(1))willbeselectedasthebestapproximated transformationofri.informulatingthetransformationforribasedonitsbestdomainblock, wherezdenotesthepixelintensityatposition(x;y),siisthecontrastscaling,andoiisthe transformation.inmatrixform,wicanbeexpressedasfollows: luminanceoset. wi264xyz375=264aibi0 Toregeneratetheoriginalimage,onlythetransformationandthepositionsoftherangeand cidi0 00si375264xyz ei fi oi375 (2) delity[fis95]. domainneedtobetransmittedtothedecoder.thefractalcodeoftheimage,iterativelyapplied toanyinitialimage,willgenerateasimulatedversionoftheoriginalimage.toensureconvergence 1Encodingsmadewithlowertolerancewillhavebetterdelity,andthosewithhighertolerancewillhaveworse 4

5 Figure1:(a)Originalimage;(b)Rangepartitionsoftheimage;(c)Rangepartitionsoverthe originalimage. atthedecoder,theunionofalltransformationsforthewholeimagemustbecontractive.fractal imagecompressionislossy[mah94]. (b) (c) havefocusedonreducingthecomplexityofencodingorincreasingdecodingspeed[lor94]. angular,andtriangularpartitionsoftherangeblockstoimproveimagedelity.otherresearchers [FJB91]ofadaptivemethodsintheencodingprocess.Theyproposedtheuseofquad-tree,rect- enhancetheinitialconcept.worthyofmentionistheintroductionbyfisher,jacob,andboss Figure1illustratesarangepartitioningofanimage.AsshowninFigure1(b),therange SinceJacquin'sinitialworkonfractalimagecoding,manyextensionshavebeenproposedto imagedata.aspointedoutin[fis95],mostnaturallyoccurringimagescanbecompressedbytaking 2.2Observations Thefractalimagecompressionapproachexploitsself-similaritywithintheimagetocompactthe partitioningoftheimagecloselyapproximatestheoriginalimage. similaritycomparisonbetweenimages. compactimagedataproposedinthefractalimagecompressioncanbeextendedtoperformthe advantageofthistypeofself-similarity.weobservedthatthestrategyofusingself-similarityto is,tobestsimulateriwithinagiventolerance.ifthereexistssuchablock,wethenhaveoneof existsablockd0jwithinm2whichcanbeusedtoperformthesamefunctionasthatofdj,that thefollowingsituations: withinagiventolerance.consideranotherimagem2.wecandeterminewhetherornotthere thereducedblockofdjthroughaveragingpixelsoersasimilaritymeasurebetweenthetwoblocks ConsiderarangeRianditsdomainDjinanimageM1.ThedistanceDistibetweenRiand M2hasasubimagethatisalsoasubimageinM1whichcontainsbothRiandDj;or M2hasablockwhichcontainstexturethatissimilartoRiinM1withinagiventolerance. 5

6 inimages. basisoftheirtexturesimilarityinthefollowingsection. transformationthroughaveragingpixelstakesintoaccountnoiseanddistortionthatmightappear thedomaind0jselectedfromm2ismeasuredthroughanetransformationofd0jtori.theane Applyingtheseobservations,weintroduceaclusteringapproachtoclassifyingimagesonthe Notethat,inthesecondsituationgivenabove,thedegreeofsimilaritybetweenRiinM1and visualdatabase.wewillrstintroduceajointfractalcodingapproachbetweenimagesandthen 3Fractal-BasedClustering 3.1JointFractalCoding discusstheapplicationofthisapproachtotheclusteringofimages. Inthissection,wewillpresentafractal-basedclusteringapproachtotheclassicationofalarge thenlikelyresultinarangeintheformerbeingbestapproximatedbyadomaininthelatter.if thetwoimagesarecloselyrelatedaccordingtoagivenmetric,wecouldthenselectmanysimilar bothm1andm2.thesimilarityintexturebetweenaportionofm1andaportionofm2will bothm1andm2.thefractalencoderapproximatestherangesofm1bydomainsthatliewithin betweentwoimagesm1andm2canbeidentiedbyperformingthefractalcodingofm1using LetM1andM2betwoimages.BasedontheobservationsgiveninSection2.2,thesimilarity domainblockinm2asthebesttransformeddomainforarangeinm1. featuresfromm2ratherthanfromm1intheformulationofthecontractivetransformationsof M1.Thesimilarityofthetwoimagescanthusbeassessedbynotingthefrequentchoiceofa intexture,thebestdomainblockforricouldbefoundinm2ratherthaninm1.letdist1i thebestapproximateddomainblockforriinbothm1andm2.ifthetwoimagesaresimilar 88squareblocks.ThejointfractalcodingofM1withrespecttoM2indicatesthat,duringthe processoffractalcodingforeachrangeblockriinm1,thejointfractalcodingprocedureseeks images.m1ispartitionedinto44size-xedsquarerangeblocks.thedomainsizeisxedat Weshallnowdeneajointfractalcodingprocedurebetweenimages.LetM1andM2betwo denotetheminimumrmsdistancebetweenrianditsdomainblockfoundwithinm2.let>0 beagiventolerancelevelfordeterminingtheclosenessbetweenm1andm2.iftheformulabelow denotetheminimumrmsdistancebetweenrianditsdomainblockfoundwithinm1anddist2i holds: jdist1i?dist2ij<; 6 (3)

7 wethenchoosethedomainblockinm2asthebestapproximationdomainblockofri. isselected.wetermsuchdomainsthatarechosenfromm2jointdomains.assumethatthe M2arecloselyrelatedintexture,manydomainblocksfromM2mightthen,followingtheselected criterion,bechosen. domainblocksinm2alwayshavehigherselectionprioritythanthoseinm1.clearly,ifm1and AdomainblockineitherM1orM2thatistheminimumrmsdistancefromtherangeblock imagedatabaseandasetoficonicimages,theimagesinthedatabasecanbecategorizedinclusters onthebasisoftheirsimilaritytotheiconicimages.thisprocedurecanbeaccomplishedasfollows. Foreachiconicimage,thejointfractalcodingprocedureapproximatestherangesoftheiconic imagebydomainsthatliewithinbothitselfandeachdatabaseimage.thenumberofdomains thatarechosenfromeachdatabaseimagefortheiconicimagecanthenbeobtained.thesimilarity Thejointfractalcodingtechniqueprovidesarobustapproachtoclusteringimages.Givenan ablockinthedatabaseimageasthedomainfortheiconicimage.thus,thisnumberplaysan oftheiconicimageandthedatabaseimagecanthenbeassessedbynotingthefrequentchoiceof importantroleondeterminingthesimilaritybetweentheiconicandthedatabaseimages.detailed discussiononsimilaritymeasurementwillbegiveninsection4. imageclusteringtechniqueinvisualdatabases,oneshouldusethisimageclusteringtechniqueto categorizedatabaseimageso-lineonthebasisofasetoficonicimages. 3.2Experiments Notethatthejointfractalcodingapproachistime-consuming.Thus,toapplytheproposed Wewillnowpresenttheexperimentalresultsconductedforthejointfractalcodingofimages.A images.table1presentstheresultsofexperimentsinvolvingjointfractalcodingbetweeneach imageat3232pixelsandthedatabaseimagesizeat9696pixels. testbedoftheimagedatabasehasbeenconstructedfrombrodatztextureimagesandtheirmixed iconicimageandeachdatabaseimagegiveninfigure2.theseresultsindicatethat,foragiven variants[bro66].subimageswereselectedfrombrodatztextureimagesetandusedasiconic images.fortheexperimentalresultspresentedinthiscontext,wehavesetthesizeoftheiconic iconicimage,amajorityofmatchingdomainblockscanbefoundinasimilardatabaseimage, whilerelativelyfewmatchingdomainblockscanbefoundinanon-similardatabaseimage.let thejointdomainblockselectionratebetheratioofthenumberofdomainsfoundinthedatabase Figure2presentsveiconicimages,eachofwhichisassociatedwiththreesimilardatabase blockselectionratesof78.12%,90.62%,and90.62%,respectively.however,domainblockswere imagetothetotalnumberofrangeblocksintheicon.considericonicimageicon(a).domain blocksforthisimagewerefoundinsimilarimagesb004j,b004k,andb004latthejointdomain 7

8 Icon(a) B008m B004j B004k B008n B004l Icon(b) B034b B034p B034t B008p Icon(d) Icon(c) B079a B082i B082j B079b B082k B079c Icon(e)Figure2:IconicanddatabaseimagesfromBrodatzAlbum. 8

9 foundinnon-similarimagesb034b,b034p,andb034tatratesofonly0.00%,0.00%,and1.56%, respectively.notethat,althoughdatabaseimagesb008m,b008n,andb008parealsodissimilar thatthesetwogroupsofimageshavesomewhatsimilartexture. rates,withimagesb004j,b004k,andb004lalsohavingrelativelyhighrates.thisagainindicates closelyrelatedtob004j,b004k,andb004lthantob034b,b034p,andb034t.similarly,taking Icon(d)astheiconicimage,weseethatimagesB079a,B079b,andB079chavethehighestselection toicon(a),thejointdomainblockchosenratesfortheseimagesareslightlyhigherthanthosefor B034b,B034p,andB034t.AsindicatedinFigure2,imagesB008m,B008n,andB008paremore TestImageIcon(a)Icon(b)Icon(c)Icon(d)Icon(e) B004k B004l B008m B008n B008p 78.12%0.00%0.00%65.62%45.31% B034b 90.62%0.00%0.00%67.19%37.50% B034p 90.62%0.00%0.00%70.31%26.56% B034t 10.94%60.94%0.00%15.62%39.06% B079a 17.19%75.00%0.00%15.62%43.75% 7.81%51.56%0.00%7.81%21.88% B079b B079c 39.56%0.00%0.00%75.00%12.50% 0.00%0.00%90.62%0.00%0.00% B082i 39.06%0.00%0.00%71.88%14.06% 53.12%0.00%0.00%85.94%14.06% 0.00%0.00%85.94%0.00%0.00% B082j 18.75%3.12%0.00%17.19%87.50% 21.88%0.00%0.00%17.19%87.50% 1.56%0.00%100.00%0.00%0.00% shallnowdiscusstheeectivenessoftheproposedapproachwhenappliedtoimagesofheterogeneous Theexperimentsdescribedabovewereconductedonimagesoffairlyhomogeneoustexture.We B082kTable1:Similaritycomparisonbetweenimages 28.12%1.56%0.00%18.75%89.06% relativelyfewmatchingdomainblockswillbefoundinadatabaseimagethatdoesnotcontain foundinadatabaseimagewhichcontainsasubimagesimilartothegiveniconicimage.however, resultsindicatethat,foragiveniconicimage,anacceptablenumberofjointdomainblockscanbe texture.figure3presentsfouriconicimages,eachofwhichisassociatedwithvedatabaseimages experimentalresultsofjointfractalcodingbetweentheiconicimageandthedatabaseimage.these ofmixedtexture.thematchingrateundereachdatabaseimageintheguredemonstratesthe 9

10 (a) suchasubimage. (b) 98.44% 93.75% 78.12% 68.75% 39.06% (c) 90.62% 81.25% 81.25% 15.62% 10.94% (d) % % 93.75% 12.50% 3.12% ontheircontentsimilarities.severalsimilaritymetricsneedbeidentiedtobeusedforsucha applyingthejointfractalcodingapproachtotheclassicationoflargevolumeofvisualdatabased SubstantialexperimentsperformedonBrodatztexturetestbeddemonstratethepotentialof % Figure3:Iconicimagesanddatabaseimageswithmixedtexture % 90.62% 85.94% 28.12% 4SimilarityMeasurementforClassifyingVisualData purpose.wewilldiscusstheseissuesinthenextsection. measurementonalargevisualdatabase.insuchanenvironment,genericsimilaritymeasurement Inthissection,wewilldeveloptheparametersthatcanbeusedtoprovideeectivesimilarity 10

11 isnecessarytoclassifyvarioustypesofnaturalimages. thenumberofjointdomainsselectedfromm2isrelativelysmallbuttheaverageminimumrms similartom1.additionally,thedegreeoftheminimumrmsdistanceplaysanimportantrole.if numberofjointdomainsselectedfromm2isanimportantfactorindeterminingwhetherm2is IntheprocessofjointfractalcodingbetweenimagesM1andM2discussedinSection3.1,the 4.1SimilarityMetrics andtheaveragedierencebetweentheminimumrmsdistancesforthosedomainblocksfoundin M1itselfandinM2. thusbedeterminedbasedonseveralfactorsincludingthenumberofdomainsfoundinm2form1 maystillconsiderthetwoimagestobesimilar.thesimilarityratebetweenm1andm2should distancebetweenrangeblocksandthecorrespondingtransformeddomainblocksisfairlylow,we inm2isd2(ri).wedenethea-factorofsimilaritybetweenm1andm2asfollows: jointfractalcodingform1bem.foreachrangeblockriinm1,assumethatthermsdistance forthebestdomainfoundinm1itselfisd1(ri)andthermsdistanceforthebestdomainfound LetthetotalnumberofrangeblocksinM1benandthenumberofdomainsfoundinM2using ofthermsdistancesbetweenthebestdomainsfoundinm1andm2forthoserangeblocksof domainsinm1.wehave M1whichchoosetheirdomainswithinM1.LetS1bethesetofrangeblockswhichchoosetheir WedenetheB-factorofsimilaritybetweenM1andM2tobethenormalizedaveragedierence A=mn: (4) ofthermsdistancesbetweenthebestdomainsfoundinm1andm2forthoserangeblocksof B=8><>:q(PRi2S1(d1(Ri)?d2(Ri))2)=(n?m) domainsinm2.wehavec=8><>:q(pri2s2(d1(ri)?d2(ri))2)=m M1whichchoosetheirdomainswithinM2.LetS2bethesetofrangeblockswhichchoosetheir WedenetheC-factorofsimilaritybetweenM1andM2tobethenormalizedaveragedierence 0maxRi2S1(jd1(Ri)?d2(Ri)j) ifm6=n, ifm=n. (5) between0and1,allthreesimilarityfactorsa,b,andcareintherangeof[0;1]. Asmin(4)isalwayslessthanorequaltonandbothBin(5)andCin(6)arenormalizedtobe 0maxRi2S2(jd1(Ri)?d2(Ri)j)ifm6=0, 11 ifm=0. (6)

12 large,thenthebestapproximateddomainsarefoundwithinm1inahighdegree.asaresult,the ratebetweenthetwoimagesshouldbehigh.bhasnegativeeectonthesimilarityrate.ifbis similarityratebetweenm1andm2shouldberelativelylow.incontrast,chaspositiveeecton islarge,thenwecanassumethatmanydomainsinm2aresimilartom1.thus,thesimilarity thesimilarityrate. Wenowdiscussthecalculationofthesimilarityratebetweenthetwoimages.Informally,ifA where,and(0;;1and++=1)aretheparametersthatdeterminetheweight proposethefollowingformulatocalculatethesimilarityratebetweenimages: Letalpha,beta,andgamma(denoted;,and,respectively)beweightingparameters.We ofparticipatingfactorsinthesimilarityrate. chooseasetofimagesfrombrodatzcollect[bro66]asoursampleimagedatabase.theseimages Thechoicesofparameters,,shouldbedecidedbasedonthenatureofapplications.We SimRate=A+(1?B)+C; (7) aredividedintogroupsaccordingtoselectediconicimages,witheachgrouphaving10similar imagestoitsiconicimage. imagesretrievedtothatofalltheimagesretrieved.moreformally,ifaisthesetofrelevantimages intermsofprecision[vr81].asstatedin[sc95],precisionistheratioofthenumberofrelevant rateforeachpairofaniconicimageandadatabaseimage;(3)foreachiconicimage,selectthe iconicimageandadatabaseimage;(2)givendierentparameters,and,calculatethesimilarity top10databaseimageswithhighestsimilarityrates.wethenevaluatetheretrievaleectiveness Theexperimentsincludethefollowingsteps:(1)calculatethevaluesofA,B,andCforeach probability:precision=p(ajb).inourexperiments,weuse: indatabaseandbisthesetofretrievedimages,thenprecisioniscalculatedbytheconditional higherandandareanynumberssatisfying++=1and0;1.thissuggests Icon006,weseethatwhenthevalueofisintherangeof[0:5;0:8]andisintherangeof[0:0;0:1], theimportanceofthenumberofdomainsfoundinthedatabaseimageshouldbeemphasized.for InFigure4,foriconIcon001,weseethatthehighestprecisionis80%whenreaches0:6or precision=retrieved?relevant Total?retrieved: (8) theprecisionreachesthehighestnumberpossible.foricon054,weseethatwhenisintherange of[0:5;1:0]andisin[0:0;0:5],theprecisionreachesthehighestnumberpossible.foricon065, whenisintherange[0:4;0:7],theprecisionreachesthehighest. indicatethatisthemostimportantfactoramongthethreeparametersandshouldbesetinthe Followingalargevolumeofexperimentsconductedonvariousvaluesof,,and,theresults 12

13 Precision(%) Query Image: Icon001 (a) beta= 0.00 beta= 0.10 beta= 0.20 beta= 0.30 beta= 0.40 beta= 0.50 beta= 0.60 beta= 0.70 beta= 0.80 beta= 0.90 beta= 1.00 Alpha Precision(%) Query Image: Icon006 (b) beta= 0.00 beta= 0.10 beta= 0.20 beta= 0.30 beta= 0.40 beta= 0.50 beta= 0.60 beta= 0.70 beta= 0.80 beta= 0.90 beta= 1.00 Alpha Precision(%) Query Image: Icon054 Query Image: Icon065 Precision(%) beta= 0.00 beta= beta= beta= beta= 0.40 beta= beta= Figure4:Retrievalprecisionandparametersforsimilaritymeasure. (c) beta= beta= 0.80 beta= beta= (d) Alpha beta= 0.00 beta= 0.10 beta= 0.20 beta= 0.30 beta= 0.40 beta= 0.50 beta= 0.60 beta= 0.70 beta= 0.80 beta= 0.90 beta= 1.00 Alpha

14 Icon (a) rangeof[0:5;1:0],shouldbechosenbelow0.2,andshouldalsobeunder0.3.specicvaluesfor theparametersshouldbedecidedempirically Figure5:VisTexdatabaseimagesclustering (b) fromimagecuts.fortheexperimentalresultspresentedinthiscontext,wehavesetthesizeofthe andsimilaritymetricsgivenabove.atestbedoftheimagedatabasehasbeenconstructedfrom thevisteximagedatabaseobtainedfromthemitmedialab.iconicimagesarerandomlyselected Wehaveconductedcomprehensiveexperimentsonvisualdataclusteringbasedontheapproach 4.2Icon-BasedClustering Intherstset,wechoose,,andtobe0.7,0.2,and0.1,respectively.Fortheicongiven infigure5(a),theclusteringtechniquesandsimilaritymeasurementweretestedonthevistex iconicimageat3232pixelsandthedatabaseimagesizeat128128pixels. database.figure5(b)illustrates10databaseimagesandtheirsimilarityratesbasedonformula (7).AscanbeobservedfromFigure5,thesimilarityratesbetweenthedatabaseimagesandthe Wenowpresentfoursetsofexperimentsonthesimilaritybetweeniconsanddatabaseimages. icontendtodecreaseasthefeaturesofthedatabaseimagesgetlesssimilartothatofthegiven 14

15 Icon (a) icon.infact,all\building"imageswithsimilardesignfeaturestotheiconareidentiedattop similarityrates.imagescontaining\tiles"docontainsomesimilarfeaturestothegivenicon,but 0.61 Figure6:VisTexdatabaseimagesclustering (b) only0.19,whichisatthelowest. FortheicongiveninFigure6(a),theclusteringtechniquesandsimilaritymeasurementweretested onformula(7). onthevistexdatabase.figure6(b)illustrates10databaseimagesandtheirsimilarityratesbased theimagecontaining\sand"hasfewfeaturessimilartothegiveniconandthus,similarityrateis Forthesecondsetofexperiments,wechoose,,andtobe1.0,0.0,and0.0,respectively. theicongiveninfigure7(a),theclusteringtechniquesandsimilaritymeasurementweretestedon tendtodecreaseasthefeaturesofdatabaseimagesgetlesssimilartothatofthegivenicon.also, all\water"imageswithsimilardesignfeaturestotheiconareidentiedattopsimilarityrates. thevistexdatabase.figure7(b)illustrates10databaseimagesandtheirsimilarityratesbased Forthethirdsetofexperiments,wechoose,,andtobe0.6,0.2,and0.2,respectively.For Similartotherstsetofexperiments,ascanbeobservedfromFigure6,thesimilarityrates onformula(7). AsalsocanbeobservedfromFigure7,thesimilarityratestendtodecreaseasthefeatures 15

16 Icon (a) Figure7:VisTexdatabaseimagesclustering (b)

17 Icon (a) ofdatabaseimagesgetlesssimilartothatofthegivenicon.also,allimageswithsimilardesign featurestotheiconareidentiedattopsimilarityrates Figure8:VisTexdatabaseimagesclustering (b) theicongiveninfigure8(a),theclusteringtechniquesandsimilaritymeasurementweretestedon onformula(7). thevistexdatabase.figure8(b)illustrates10databaseimagesandtheirsimilarityratesbased databaseimagesgetlesssimilartothatofthegivenicon.also,allimageswithsimilardesign Forthefourthsetofexperiments,wechoose,,andtobe1.0,0.0,and0.0,respectively.For featurestotheiconareidentiedattopsimilarityrates. AscanbeobservedfromFigure8,thesimilarityratestendtodecreaseasthefeaturesof retrieval.forourexperimentalneeds,weclassifyvisualdataintothreecategories:highsimilarity mediumdegreesofsimilaritygroupswouldbesignicantforsupportingcontent-basedvisualdata degreesofsimilaritygroupsbasedontheirsimilarityratestoagivenicon.usually,bothhighand group(similarityrate0:7),mediumsimilaritygroup(similarityrateisbetween0:69and0:4), andlowsimilaritygroup(similarityrateislessthan0:39).notethatanimagemaybelongto Basedontheseexperimentalresults,wecanseethatvisualdatacanbecategorizedintodierent severalgroupsfordierenticons. 17

18 5Discussion Substantialresearchhasbeendirectedtowardthesupportofecientindexingtechniquesbasedon teringapproachesarealsoconsidered. 5.1ClusteringandIndexing Inthissection,wediscusstherelationshipbetweenclusteringandindexing.Otherpotentialclus- featurevectorsofimages[bpj93,sna+95,sc94b,srf87,bkss90,ljf94].thefeaturevectorof animagerepresentsvariousfeaturesoftheimage.mostsuccessfulapproachesarebasedontexture, shape,andcolorfeaturestoformulatefeaturevectors.givenfeaturevectorsofimages,indexing algorithmssuchasr-tree,r+-tree,andtv-tree[srf87,bkss90,ljf94]havebeenproposedto [ZCAM96].Featurevectorsofimagesareconstructedfromwavelettransforms,whicharethen supportecientaccessestovisualdatabases. usedtodistinguishimagesthroughmeasuringdistancesbetweenfeaturevectors.ourexperimental resultsdemonstratethatthisapproachcanbeeectivelyusedtoperformcontent-basedsimilarity comparisonbetweenimages.however,wealsoencounteracriticalproblemthatsomesemantically irrelevantvisualdatamayhavefeaturevectorsthatfallwithinaverysmalldistance.thus,clusteringonfeaturespacesometimesmaynotprovidesatisfactorysolutions.amoreeectiveclustering Wehaveconductedexperimentalresearchonvisualdataretrievalbasedonwavelettransforms approachisneededtobeintegratedintoindexingtechniquesforecientretrievalofvisualdata. technique,visualdataclassicationbasedonasetoficonicimagescansupportanoveldivision isnotintendedtobeveryaccuratebuttoprovideagoodapproximatecategorizationofvisual databasedonagivensimilaritymeasurement.weobservethat,usingthejointfractalcoding ofvisualdatabasedontheirsimilaritytotheiconicimages.withineachcategoryofimages, classifyvisualdatabasedontheirsimilarityfeatures.inalargevisualdatabase,suchaclassication Theincorporationofclusteringintoindexingtechniquescanprovideaneectivemethodto featurevectorsofimagescanthenbegeneratedusingthetechniquesproposedinseveralimage techniquecanbeselectedfromvariousexistingindexingtechniquesthatarethevariationsofrtree[srf87,bkss90,ljf94].theseindexingtechniquescanbeappliedtothefeaturevectors theimageswithineachcluster.anecientvisualdataretrievalsystemwillthenbeestablished basedonthesetools. retrievalapproachesbasedoncolor,texture,andshape[bpj93,sc95,sna+95].anovelindexing Thirdly,thevariationofR-treetechniqueisusedagainoneachclustertoconstructanindexon ofimages.ahierarchicalindexstructurecanthenbeestablishedfortheentirevisualdatabase. clusteringapproachgivenaboveisappliedtoclassifythevisualdatabasedonthegivenicons. Firstly,avariationofR-treeisusedtoconstructanindexontheselectedicons.Secondly,the 18

19 beperformedontheiconicimagesbycomparingthedistanceoffeaturevectorsbetweenthequery imageandicons.asubsetoftheiconicimageswillbeidentiedasthesubdirectoriesforfurther retrieval.foralargevisualdatabase,thisstepcansubstantiallynarrowthesearchtomorerelevant Theselectionofsuchiconsisapplication-oriented.Foragivenqueryimage,theretrievalwillrst givenvisualdatabase,weestablishahierarchicalindexstructurebasedonasetofselectedicons. Theproposedclusteringandindexingapproachcanecientlysupportvisualqueries.Fora 5.2OtherClusteringApproaches areclassiedintodierentdegreesofgroups,somedegreeofsimilarityretrievalcanbesupported bysearchsimilarimagesfromdierentgroups. images.thequerycanthenbemoreecientlyperformedontheselectedclusters.asvisualdata Oneadvantageoftheproposedclusteringapproachisthattheself-similarityfoundwithintheimage isutilizedasthethresholdtomeasurethesimilaritybetweenblocksindierentimages.sucha similaritythresholdisautomaticallyadjustedfordierentblocksintheimage.experimentshave notclearlydened,itisunclearatthispointwhetherthereisanapproachwhichcanautomatically posesignicantproblems.sincethedegreeofsimilaritybetweenimagesontheirpixelformatis calculationsusingidenticalsizedrangesanddomains.however,similaritythresholdselectionwill iconsanddatabaseimagesforclusteringcanbeperformedontheirjpegcoecientsorrms demonstratedthatthisapproachishighlyeective. determineasimilaritythresholdtoselectthebestdomainsindatabaseimagesfortherangesin Otherclusteringapproachesarepossible.Forexample,thesimilaritycomparisonbetween 6ConclusionandFutureResearch eachicon.theseissuesmeritdetaileddiscussionsandwillnotbefurtherpursuedinthispaper. imagecoding.ahierarchicalclusteringandindexingstructurecanbedesignedforlargevisual Inthispaper,wehavepresentedapproachestotheclusteringofimagedatabasedonthefractal databases.theproposedimageclusteringapproachusesajointfractalcodingtechniquebetween databaseapplications. imagecontentforretrieval.thus,theproposedapproachisespeciallyimportanttolargevisual opportunitytocompactimagedata,andthispropertycanalsobeutilizedtoidentifythesimilarity ourexperimentalresults,theself-transformationpropertyoffractalimagecompressionoersthe betweenimages.thisprocessoperatesautomatically,withnohumanassistancerequiredtoextract images,whichcanbeusedtodeterminethedegreeoftheirsimilarity.aswehaveshownin behighlyeective.additionalresearchandexperimentalworkmustbedonetoecientlyincor- Initialexperimentalresultsconductedonnaturalimageshavedemonstratedthisapproachto 19

20 References [ACF+93]ManishArya,WilliamCody,ChristosFaloutsos,JoelRichardson,andArthurToga. comprehensiveclusteringsystemforlargevisualdatabasescanbeeectivelydeveloped. poratethesetechniquesintodomainspecicvisualdatabaseanddigitallibraryapplications.a [BKSS90]N.Beckmann,H.P.Kriegel,R.Schneider,andB.Seeger.TheR*-tree:anecientand RobustAccessMethodforPointsandRectangles.InProceedingsofACM-SIGMOD QBISM:APrototype3-DMedicalImageDatabaseSystem.IEEEDataEngineering Bulletin,16(1):38{42,1993. InternationalConferenceonManagementofData,pages322{331,AtlanticCity,NJ, [Bro66]P.Brodatz.Textures:APhotographicAlbumforArtistsandDesigners.Dover,New [BPJ93]J.R.Bach,S.Paul,andR.Jain.AVisualInformationManagementSystemforthe May1990. [BS88]MichaelF.BarnsleyandAlanD.Sloan.ABetterWaytoCompressImages.BYTE, York, (4):619{628,1993. InteractiveRetrievalofFaces.IEEETransactionsonKnowledgeandDataEngineering, [CYDA88]S.K.Chang,C.W.Yan,DonaldC.Dimitro,andTimothyArndt.AnIntelligent pages215{223,1988. [FJB91]Y.Fisher,E.W.Jacobs,andR.D.Boss.IteratedTransformationImageCompression. [Fis95]Y.Fisher.FractalImageCompression:TheoryandApplication.Springer-Verlag,1995. ImageDatabaseSystem.IEEETransactiononSoftwareEngineering,14(5):681{688, TechnicalReportTR-1408,NavalOceansSystemsCenter,SanDiego,CA,1991. May1988. [GS] [FL92]Y.FisherandA.F.Lawrance.FractalImageCompressionformassstorageapplications. [HJS94]MichaelL.Hilton,BjornD.Jawerth,andAyanSengupta.CompressingStillandMoving SPIEImageStorageandRetrievalSystems,1662,1992. T.GeversandA.W.M.Smeulders.AnApproachtoImageRetrievalforImage Databases,volume720.LectureNotesinComputerScience,Springer-Verlag,Berlin. ImageswithWavelets.MultimediaSystems,2(5):218{227,December

21 [HLHC92]Tai-YuanHou,PeiyaLiu,ArdingHsu,andMing-YeeChiu.MedicalImageRetrieval [Jac92]ArnaudE.Jacquin.ImageCodingBasedonaFractalTheoryofIteratedContractive byspatialfeatures.inieeeconferenceonsystems,man,andcybernetics,1992. [LJF94]K.-I.Lin,H.V.Jagadish,andC.Faloutsos.TheTV-Tree:anIndexStructurefor [Jac93]ArnaudE.Jacquin.FractalImageCoding:AReview.ProceedingsoftheIEEE,81(10), High-DimentionalData.TheVLDBJournal,3(4):517{542,October1994. ImageTransformations.IEEETransactionsonImageProcessing,1(1),1992. [MAH94]M.Gharavi-AlkhansariandT.S.Huang.Fractal-BasedTechniquesforaGeneralized [NBE+93]W.Niblack,R.Barker,W.Equitz,M.Flickner,E.Glasman,D.Petkovic,P.Yanker, [LOR94]S.Lepsqy,G.E.Oien,andT.A.Ramstad.AttractorImageCompressionwithaFast ImageCodingMethod.InProceedingsIEEEICIP,1994. non-iterativedecodingalgorithm.inproceedingsoficassp-93,1994. [SC94a]J.R.SmithandS.Chang.TransformFeaturesForTextureClassicationandDiscriminationinLargeImageDatabases.InProceedingsIEEEICIP,1994. October1994. Query.InProceedingsofACMMultimedia94,pages279{286,SanFrancisco,California, [RS91]F.RabittiandP.Savino.AutomaticImageIndexationandRetrieval.InConference onintelligenttextandimagehandling,1991. texture,andshape.technicalreport,ibmtechnicalreport,1993. C.Faloutsos,andG.Taubin.Theqbicproject:Queryingimagesbycontentusingcolor, [SC94b]JohnR.SmithandShih-FuChang.Quad-TreeSegmentationforTexture-BasedImage [SC95]JohnR.SmithandShih-FuChang.AutomaticImageRetrievalUsingColorandTexture.Technicalreport,ColumbiaUniversity,1995. [Slo94]AlanD.Sloan.RetrievingDatabaseContentsbyImageRecognition:NewFractal [SH94]DietmarSaupeandRaoufHamzaoui.Aguidedtourofthefractalimagecompression literature.inacmsiggraph'94coursenotes,newdirectionsforfractalmodelling incomputergraphics,july1994. Power.AdvancedImaging,9(5):26{30,

22 [SNA+95]M.FlicknerH.Sawhney,W.Niblack,J.Ashley,Q.Huang,B.Dom,M.Gorkani, [SRF87]T.Sellis,N.Roussopoulos,andC.Faloutsos.TheR+-tree:ADynamicIndexof [TMY78]HideyukiTamura,ShunjiMori,andTakashiYamawaki.TextureFeaturesCorrespondingtoVisualPerception.IEEETransactionsonSystems,Man,andCybernetics,8(6), MultidimensionalObjects.InProceedingsofthe18thVLDBconference,1987. J.Hafner,D.Lee,D.Petkovic,D.Steele,andP.Yanker.QuerybyImageandVideo Content:TheQBICSystem.IEEEComputer,28(9),1995. [TPF+91]A.Turtur,F.Prampolini,M.Fantini,R.Guarda,andM.A.Imperato.IDB:AnImage [vr81]cornelisj.vanrijsbergen.retrievaleectiveness.inkarensparckjones,editor, June1978. [Wal91]GregoryK.Wallace.TheJPEGStillPictureCompressionStandard.Communications DatabaseSystem.IBMJournalofResearchandDevelopment,35(1):88{96,January [WN94]JianKangWuandArcotDesaiNarasimhalu.IdentifyingFacesUsingMutipleRetrievals.IEEEMultimedia,1(2):27{38,1994. ImageDatabaseSystems.InProceedingsoftheSPIEConferenceonDigitalImage StorageandArchivingSystems,Philadelphia,October1995. InformationRetrievalExperiment,pages32{43.Butterworths,1981. [ZCA95a]AidongZhang,BiaoCheng,andRajAcharya.AnApproachtoQuery-by-texturein oftheacm,34(4):30{44,1991. [ZCA95b]AidongZhang,BiaoCheng,andRajAcharya.Texture-basedImageRetrievalinImage [ZCAM96]AidongZhang,BiaoCheng,RajAcharya,andRaghuMenon.ComparisonofWavelet TransformsandFractalCodinginTexture-basedImageRetrieval.InProceedingsof invitedpaper. andexpertsystemsapplications(dexa),london,unitedkingdom,september1995. DatabaseSystems.InProceedingsoftheSixthInternationalWorkshoponDatabase thespieconferenceonvisualdataexplorationandanalysisiii,sanjose,january