A graph based framework for the definition of tools dealing with sparse and irregular distributed data-structures

Transcription

1 A graph based framework for the definition of tools dealing with sparse and irregular distributed data-structures Serge Chaumette, Jean-Michel Lepine, Franck Rubi To cite this version: Serge Chaumette, Jean-Michel Lepine, Franck Rubi. A graph based framework for the definition of tools dealing with sparse and irregular distributed data-structures. 3rd International Workshop on High-Level Programming Models and Supportive Environments (HIPS 98), Mar 998, Orlando, Florida, United States. IEEE Computer Society, pp <hal-36323> HAL Id: hal Submitted on 2 Feb 29 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Agraphbasedframeworkforthedenitionoftoolsdealingwith sparseandirregulardistributeddata-structures SergeChaumette,Jean-MichelL pineandfranckrubi Tel.:(+335) Fax:(+335) Lib ration,3345talence,france. LaBRI,LaboratoireBordelaisdeRechercheenInformatique,Universit BordeauxI,35Coursdela Keywords ProgrammingEnvironments,DataParallelism,Tools,Visualization,SparseDataStructures,DistributedData Structures,IrregularDataStructures,Graphs Abstract Industrialapplicationsmainlyusestandarddatastructuressuchasmatrices,butmostofthetimeprovideaspecic asmatricescomingfromthedomainofniteelements[23].thegapbetweentheimplementationandtheabstract cicimplementationsareespeciallyoftenlyusedwhendealingwithlargesparseandirregulardata-structures,such problem-orientedimplementation,e.g.compressedsparsecolumn(csc)seeforinstancesparskit[2].spe- datastructureitimplementsisevenbiggerwhenconsideringdataparallelapplicationswheredatastructuresare distributedoveranetworkofprocessors.hencethereisaneedfortoolsthatmakeitpossiblefordevelopersto visualizeboththeirdata,theirstructure,andtheoperationsthatareappliedtoit,whatevertheireectiveimplementationandtheirdistributionare.suchtoolsmustprovidehighlevelviews,i.e.abstractfromthephysical implementationtoreachthedeveloper'sview.theymustcarrythesemanticsoftheapplicationsandprovide synthesisorlteringmechanismsthatmakeitpossibletofocusonaspecicaspectoftheproblem. prototypestoolsbasedonitwhichwehavedeveloped.theresultingenvironmentiscomposedoftwolayers:the Inthispaper,wepresentaframeworkwhichwehavesetuptosupportthedevelopmentofsuchtools,and rstlayerisamodelthatwehavedenedandimplementedashighlevellibrariesthatmakeitpossibletoeciently ThisworksissupportedbytheFrenchGDR-PRCPRS.

3 areintegratedwithinagraphicalenvironmentcalledvisit[7]whichispartofthehpfitresearcheort2[7,8,9]. abstractfromtheimplementation;thesecondlayeroersprototypetoolsbuiltontopoftheselibraries.thesetools andgmd/scaiinbonn,germany.theaimofthisprojectistoprovideanintegratedhpfdevelopment HPFITisajointprojectinvolvingthreeresearchlaboratories:LIPinLyon,France,LaBRIinBordeaux,France, environmentthatsupportssparseandirregulardatastructures. Thetwomainresearchdirectionswithinthedataparallelframeworkareexpressionmodes(orlanguages) Introductionandrelatedwork andtools. ulardatastructuresindata-parallellanguages(see[4,5])ortodevelopdedicatedlibraries,toprovide Expressioniswidelystudied.Theaimofmanyinitiativesiseithertointroducesparseandirreg- SPARSEKIT[2],P-SPARSLIB[22]orPETSc[2]). programmerswithreadyimplementationsofpossiblysparsedistributeddatastructures(seeforinstance P2D2[3],Pharos[25],Paradyn[8]).Wewillnotdescribealloftheminthispaperbecausetheyareless Toolsexist,buttheymainlydealwithregular(HPF-like)datastructures(seeforinstanceEPPP[6], relatedtoourgoalwhichistodealwithirregulardatastructures(see[9]foracompletesurvey). theparadigmofdata-parallelprogramming.nevertheless,theresearchwhichhasbeendoneduring Concerningirregularitythereisstillalottobedoneintermsoftoolsthatwouldhelpuserstotackle pastyearsintheareaofmessagepassinghasprovenquitesuccessfulinprovidingsupporttoend-users own.furthermore,publicdomaintools(suchasparagraph[5])arenowbeingdeliveredeitherasfully (seeforinstancetopsys[2,3,6]).bothhardwareandsoftwarevendorssupplyenvironmentsoftheir supportedorsimplyasportedtools.thesetoolsthatwereusedwhendealingwithmessagepassing applicationscanstillbeusedwithintheframeworkofdataparallelismprovidedtheexecutionsupport isdistributed(inwhichcasethecompilertranslatesdataparallelismtomessagepassingparallelism). 2ThisworkispartlysupportedbytheCNRS-INRIAprojectReMaPandbytheFrenchGDR-PRCPRS. 2

4 Nevertheless,thereisalackofrelationshipbetweentheinformationtheyprovide(whichisintermsof messages)andthesemanticsoftheapplication(whichisintermsofdata).thismakesthemhardlyusable measurement.thereasonforthislackofadequationisthatthebehaviorofanalgorithmismostofthe toexplorethealgorithmicbehavioroftheapplication,althoughtheycanstillbeusefulforperformance timebetterunderstoodwhenconsideringtheabstractstructureitworkswith,ratherthanthephysical implementationofthisdatastructure.forinstanceifthealgorithmhandlesatreethatisimplemented usingavector,itismostprobablythecasethatthisalgorithmcanbebetterunderstoodintermsof requireinformationthatcannotalwaysbeaccessedeasily,suchasdistributionofdata.aconvenientway thetreethanintermsofthevector.oneofthereasonswhyhighleveltoolsaremissingisthatthey implementedinivd[4].thisisalwaysquiteheavyfortheprogrammer.anothermanneristohave toproceedistorelyontheusertosupplytheseinformation.thisistheapproachwhichisforinstance timeasthelanguagelibrariesthemselves.thisistheapproachwhichisachievedinoneofptools librariesthatinstrumentthebasicsofthelanguageandwhicharelinkedtotheapplicationatthesame projectscalleddistributedarrayqueryandvisualization(daqv[7]).thisisnotstraightforwardly portable. data-structuresinsideapplications:wewanttoprovidetoolsdealingwithsuchdatastructuresatahigh Ourapproachisquitedierent.Ouraimisnottoprovidesupportforusingsparseandirregular librariestoabstractfromimplementationanddistributionofdatastructures.theselibrariesareportable levelofabstraction.furthermore,wewanttomakeiteasytodevelopsuchtools.hence,werstdesigned andrelyverylittleontheprogrammer.wethendevelopedtoolsbasedontheselibraries.inthispaper wepresentboththeconceptsoftheselibrariesandthetoolsbasedonthem. weproposeinsection2.section3introducesthewaywemodelizedatastructures.wethendescribethe Therestofthispaperisorganizedasfollows.Werstdescribetheoverallarchitectureofthemodel toolsbasedonourmodel.itisillustratedwithtwosoftwarecomponentswhichwehavedeveloped:data threecurrentlevelsofourmodelinsections4,5and6.section7presentsthegeneralprinciplesofthe DistributionDisplayandTraceDataDisplay.Weeventuallysketchfutureworkdirections. 3

5 Ourmodeliscomposedoffourlevels.Eachoftheselevels,i.e.implementation,abstraction,mapping 2 Generalarchitectureofthemodel andviewisagraph(gure2):.igraph:theimplementationgraph(gure(a))describestheimplementationofthedatastructure, e.g.threevectorsforacompressedsparsecolumnstorage. intermsofdataitemsandaccessfunctions,i.e.thewaytheyareaccessedwithintheapplication 2.AGraph:theAbstractionGraph(gure(b))describestheabstractdatastructuretheapplication 3.MGraph:theMappingGraphdescribestherelationshipbetweentheIGraphandtheAGraph.This developerhasinminde.g.amatrix. 4.VGraph:theViewGraph(gures(c)and(d))describeshowatoolwilleventuallyseethedata graphisabridgethatcarriesthesemanticsbetweentheimplementationandtheabstraction. Itisnotyetimplemented(seesection8),butonecanworkwiththeAGraph,whichisequivalent structuree.g.acolumnofamatrix.thislevelprovidesforsynthesisandlteringofinformation. Althoughthiswillnotbedetailedhere,themodelprovidesthesameeciencywhenworkingatthe tohavinganidentitybetweentheagraphandthevgraph. mappinggraphlevelorattheimplementationgraphlevel(seegure3). 3Wedeneadatastructureintermsofentrypointsandaccessfunctions.Thisapproachreectstheway Modelizationofadatastructure adatastructureisimplemented.itleadstoamodelizationintermsofagraph.forinstanceavector intv[]canberepresentedbyagraph,therootofwhichisv[]andthenodesarev[i]. AdatastructureDSisasetofdataitemsthatarestructuredbymeansofaccessfunctions. DenitionDatastructure where: DS=(D;d;F;e;E) 4

6 Successor function for the first vector... Entry Point... e f (a)implementation... Successor function for the third vector f (b)abstractionasamatrix (c)viewasamatrix (d)viewasacolumn disapositiveintegerrepresentingthenumberofaccessfunctions; Disthesetofdataitems; Figure:Architectureoftheframework Fisad-upleofaccessfunctions; eisapositiveintegerrepresentingthenumberofentrypoints; Eisthesetofentrypointsinthedata-structure. canthemselvesbedatastructures,i.e.graphs. Irregulardatastructurescanbedescribedusingthisframework:thenodesofthedatastructuregraph tationtheremightbedierentsetsofaccessfunctions,dependingonhowtheprogrammeraccessesthe Accessfunctionsfiexpresshowoneaccessesthebasicdataitems.Notethatforthesameimplemen- 5

7 Implementation Graph Mapping Graph Abstraction Graph View Graph Tools (Visualization, traces analysis, debug,...) Figure2:Architectureofthemodel User Efficiency Implementation Mapping Abstraction View data-structure. Figure3:Eciencyofthemodellevels Model Level Denition2Accessfunction Anaccessfunctionisafunctionoverthenodesofthedatastructuresthatbeinggivenanodeproduces anothernode. Eachaccessfunctionmakesitpossibletomovewithinadimensionofthedata-structure. f:n!n n7!f(n) whichrepresentaccessfunctions. Inotherwords,adata-structureisagraph,thenodesofwhichcontainthedataitems,theverticesof Denition3Adatastructureisagraph DS=(N;d;F;r;R) 6

8 where: Nisasetofnodes; Fisad-upleofsuccessorfunctions; disthenumberofsuccessorfunctions; risthenumberofrootsofthegraph. Risthesetofrootsofthegraph. directaccessdatastructures. Therearesomedatastructures,whereeachdataitemcanbeaccesseddirectly,likesets;wecallthem Denition4Directaccessdata-structure AdatastructureDS=(D;d;F;e;E)issaiddirectaccessifandonlyifE=D,d=,F=;and e=jdj. Insuchacaseallofthedataitemsofthedatastructurearerootsofthegraphusedtodescribeit. likenotationsformulti-dimensionalobjectsrepresentedbygraphs. In[7],weintroducefurtherdenitionsbasedonpreviousworkbyM.Alabau[],thatprovidematrix- Inthissectionwepresentanexample,theaimofwhichistoillustratehowthedenitionsofsection3 4 Describingtheimplementationdatastructure canbeusedtomodelaneectiveimplementation. connectedgraph.thenodesarethevaluesofthethreevectorsplustheentrypoint.wegivethree Theimplementationcontainsthreevectorsofdierentsizes.Anentrypointisaddedtoobtaina successorfunctions;eachfunctionallowstoaccessthenodesofoneofthethreevectors. alsobeinterpretedasacompressedsparserowimplementation,orasanyotherdatastructure.the ThisgraphmaymatchaCompressedSparseColumnimplementationofasparsematrix.Itmay interpretationofthisimplementationasagraphdoesnotdescribethesemanticsofwhatiseectively implemented.thebenetisthatthedescriptionofthisgraphbytheprogrammerisstraightforward. 7

9 Successor function for the first vector... Entry Point Figure5showsasampleofthecodetheprogrammermustwriteinordertodescribethisimplementation. Figure4:Agraphmodelizationoftheirregularstructure Successor function for third vector 5Wecallabstractiontheabstractdatastructurethattheuserwantstomanipulate.Sinceanabstraction Describingtheabstractiondata-structure aimoftheabstractionistooeraninterpretationoftheimplementation.forinstancethethreevectors isitselfadata-structure,itcanberepresentedbyanentrypointandaccessfunctions,i.e.agraph.the ofsection4canbeinterpretedasacscstorageofamatrix(seegure6). abstractiongraph. Hence,abstractingfromanimplementationconsistsinmappingtheimplementationgraphtothe 6Themappingestablishesacorrespondencebetweentheimplementationgraphandtheabstractiongraph. Mapping Itisusedtomovefromtheabstractiontotheimplementationandvice-versa. graphandanodeoftheabstractiongraph.suchanodecanonlyexistprovidedthereisanodeofthe Itisagraph,thenodesofwhicharedenedasapaircontainingbothanodeoftheimplementation implementationcorrespondingtothegivennodeoftheabstraction(theabstractionwhichisdenseusually hasmorenodesthantheimplementationwhichissparse).assumewehaveafunctionthatgivenanode 8

10 datastructure).nowonwewillcallthisfunctionabs_to_imp. oftheabstractionreturnseitheranodeoftheimplementationornull(probablyaholeinthesparse Denition5Nodes=f(abstoimp(anode);anode)=anode2agraphenodesg Wecanthendenethenodesofthemappinggraphasfollows: RemarkWecanalsodeneafunctionimp_to_abs(implementationnodetoabstractionnode).It allowsanothersymmetricdenitionofthemappinggraph.theusercandenethetwofunctions(togain eciency).oneofthesefunctionsmaybediculttowrite:itcanbelefttothetoolstoconstructit usingtheotherfunction,storingthisinformationinahashtable. 7Theframeworkpresentedaboveisbeingusedtodevelophighleveltools.Inthissectionwepresenttwo Usage prototypes. 7. Usingourlibraries,highleveltoolsworkonlywithgraphs,whichtheycanforinstancescan,usingeither Generalprinciple oftheloopsshowngures7and8. graphs.thisisusedforinstancebythetracesvisualizationtooldescribedbelow. Furthermore,thelibrarymakesitpossibletoattachadditionalinformationtoanynodeofanyofthe Thegoalofourtoolsistooerasetofsoftwarecomponentstovisualizeandanalyzethebehaviorof 7.2 Eectiveusage data-parallelprogramsintermsofthedatatheyworkwith.theframeworkpresentedaboveisbeingused viewgraphs. todevelopthesetools.theyareimplementedeitherintermsofabstractiongraphs,mappinggraphsor DDD(DataDistributionDisplay)andTDD(TraceDataDisplay).Partsofthesetoolsarecomponents Asofwriting,someofthesetoolsareavailableasprototypes.Thissectiondescribestwoofthem: 9

11 ofthehpfitproject.acompletedescriptioncanbefoundin[2].weonlypresentscreendumpshere. TheycomefromaCholeskyfactorizationofasparsesymmetricpositivedenitematrix.Theparallel machinewhichhasbeenusedtorunittocollecttracesisanibmsp2. languageandcompiledusingadaptor[]extentedwithalibrarycalleddddt(distributedderived Weinterfacedourtoolswithadataparallelsystem.ThesourceprogramiswritteninaHPF2-like Foranecientparallelexecution,aspecicirregulardistribution[]isused. DatatypeforTree).Thislibraryallowsthemanipulationofhierarchicalaccessirregulardatastructures[7]. 7.3 Figure9showsthedistributionofthedataonthevirtualprocessors(onecolorbyprocessor). DataDistributionDisplay dataitem(resp.processor).datadistributiondisplaycanbeusedbeforetheexecutionifthedistribution Usingthemouse,theusermayselectavirtualprocessor(resp.data)andvisualizethecorresponding isregularorcomputable. Theparallelapplicationisinstrumentedtogeneratetracesintermsofaccessestodataitems.Atruntime, 7.4 TraceDataDisplay TraceDataDisplayisapostmortemtoolbasedonruntimegeneratedtracesandonthemappinggraph atracecollectorrecordseventsoneachprocessor.filteringmethodsareusedtolimittheamountoftraces. ofourmodel. thecholeskyfactorization[].thiscanforinstancehelptheusertoestimatethequalityofthedata Figureshowswhichdataitemsareinvolvedinremotereadoperationsduringtherstpartof distribution. Inthispaperwehavesetupaformalapproachtothemodelizationofsparseandirregulardatastructures. 8 Conclusionandfuturework Usingthisframework,wehaveshownthatitispossibletodescribeimplementations,onlyintroducing semanticsinthemappingfromtheimplementationtotheabstraction.dependingofwhattheyaredoing,

12 toolsbuildontopofthisframeworkcaneitherusetheabstractiongraph,or,tobemoreecient,usethe mappinggraphthatmakesitpossibletoscanthedatastructureavoidingtolookatholes. ourmodel,weareprovidedwithanabstractionthatrepresentsamatrixasshowngure(a).theuser Anotherlevelofabstractionthatwestillhavetoimplementisthatofltering.Assumethatwithin maywanttoseeeitherthematrixitself,or,forinstance,agivenrowofthismatrix(seegure(b)), orevenallitemsofeachrowasauniqueitem,i.e.heneedstoltertheabstraction.thereforeweare workingonthedenitionofaviewgraphwithamappingfromtheabstractiontoit..designinglibrariesthatwouldprovidestandardabstractionsusingthismodelforstandardimplementations,likethoseofsparskitforinstance; Thenextstepswithinthisprojectare: 2.extendingthehighleveltoolsbasedonthismodel,toaddthemtoVisit[7]insidetheHPFIT 3.validatingthetoolswiththeend-userstoproposeviewsadaptedtotheirneeds; environment; 4.interfacingthisframeworktoexistingsoftwareenvironmentssuchasDAQV[7]orEMILY[24]. References []M.Alabau.Uneexpressiondesalgorithmesmassivementparall les structuresdedonn esirr guli res.th se, [2]T.Bemmerl.Anintegratedandportabletoolenvironmentforparallelcomputers.InProceedingsoftheIEEE LaBRIUniversit BORDEAUXI,September994. [3]T.BemmerlandA.Bode.Anintegratedenvironmentforprogrammingdistributedmemorymultiprocessors.InBodeA.,editor,ProceedingsoftheSecondEuropeanDistributedMemoryComputingConference InternationalConferenceonParallelProcessing(St.Charles,USA),pages553,988. [4]A.J.C.BikandH.A.G.Wijsho.Compilationtechniquesforsparsematrixcomputations.Technical (M nchen),volume487oflecturenotesincomput.sci.,pages342.springer-verlag,99. report,universityofleiden.

13 [5]A.J.C.BikandH.A.G.Wijsho.Advancedcompileroptimizationsforsparsecomputations.Journalof [6]A.Bode.Developmentsindistributedmemoryarchitectures.InProceedingsofMicrosystem'9(Bratislava, ParallelandDistributedComputing,(3):424,995. Systems,TUM-I93,SFB-BerichtNr.342/9/9A,January99,seiten6. MethodenundWerkzeugef rdienutzungparallelerrechnerarchitekturen,topsys,toolsforparallel CSSR),99.AlsoinTechnischeUniversit tm nchen,institutf rinformatik,sonderforschungsbereich342: [7]T.Brandes,S.Chaumette,M.-C.Counilh,A.Darte,J.C.Mignot,F.Desprez,andJ.Roman.HPFIT:ASet ofintegratedtoolsfortheparallelizationofapplicationsusinghighperformancefortran:partii:data StructuresVisualizationandHPFSupportforIrregularDataStructureswithHierarchicalScheme.Parallel [8]T.Brandes,S.Chaumette,M.-C.Counilh,A.Darte,J.C.Mignot,F.Desprez,andJ.Roman.HPFIT:ASet Computing,996.EditedbyJ.DongarraandB.Tourancheau. andthetranstoolenvironment.parallelcomputing,996.editedbyj.dongarraandb.tourancheau. ofintegratedtoolsfortheparallelizationofapplicationsusinghighperformancefortran:parti:hpfit [9]T.Brandes,S.Chaumette,andF.Desprez.TransTOOL:atoolforportingscienticapplicationsonparalleldistributedmemorymachines.In2ndEuropeanSchoolofComputerScience,ParallelProgramming []T.BrandesandF.Zimmermann.ADAPTOR-ATransformationToolforHPFPrograms.InK.M.Decker EnvironmentsForHighPerformanceComputing,996. andr.m.rehmann,editors,programmingenvironmentsformassivelyparalleldistributedsystems,pages []P.Charrier,FackL.,andJ.Roman.Blockdatapartitionforparallelnesteddissection.InProceedingsofthe 996.Birkh userverlag,april994. [2]S.Chaumette,F.Rubi,andLepineJ.M.Internalreport,LaBRI,Universit Bordeaux-I,997.Tobe 7thSIAMconferenceonparallelprocessingforscienticcomputing.SiamEditions,995. [3]DoreenChengandRobertHood.Aportabledebuggerforparallelanddistributedprograms.InProc.of published. [4]M.C.Hao,A.H.Karp,M.Mackey,V.Singh,andJ.Chien.On-the-yvisualizationanddebuggingofparallel Supercomputing'94,994. programs. 2

14 [5]M.T.HeathandJ.A.Etheridge.Visualizingtheperformanceofparallelprograms.IEEESoftware,8(5):2939, [6]G.Hurteau,V.VanDongen,andG.Gao.OverviewofEPPP-anEnvironmentforPortableParallelProgramming.InProceedingsofSupercomputingSymposium'94,Canada'sEighthAnnualHigh ftp://ftp.crim.ca/apar/public/papers/994/ss94-eppp.ps.gz. PerformanceComputingConference,pages927,Toronto,Ontario,June994.Alsoavailableas September99. [7]A.Malony,MaryZosel,MayJohn,AlanKarp,andDavidPresberg.PTOOLSprojectproposalDistributedArrayQueryandVisualization.Availableashttp:// [8]BartonP.Miller,MarkD.Callaghan,JonathanM.Cargille,JereyK.Hollingsworth,R.BruceIrvin, toolsforparallelanddistributedcomputersystems. KarenL.Karavanic,KrishnaKunchithapadam,andTiaNewhall.TheParadynParallelPerformanceMeasurementTools.IEEEComputer,28():674,November995.Specialissueonperformanceevaluation [9]J.LPazat.SpringSchoolondataParallelism,chapterToolsforHighPerformanceFortran:aSurvey.Springer [2]BalayS.,GroppDW.D,McInnesL.C.,andSmithB.F.ModernSoftwareToolsinScienticComputing, Verlag,996. chapterecientmanagementofparallelisminobject-orientednumericalsoftwarelibraries,pages6322. [2]Y.Saad.SPARSEKIT:ABasicToolsKitforSparseMatrixComputations,June994.Version2.Documentation. BirkhauserPress,997.Alsoavailableathttp:// [22]Y.SaadandA.V.Malevsky.P-SPARSLIB:aportablelibraryofdistributedmemorysparseiterativesolvers, [23]G.StrangandG.F.Fix.AnAnalysisoftheFiniteElementMethod.PrenticeHall, Documentation. [24]LoosT.andBramleyR. EMILY:avisualizationtoolforlargesparsematrices. Availableas [25]ThePHAROSTeam.ThePHAROSproject.Availableashttp:// 3

15 5/*composedhereof3Vectorsofdifferentsizes*/ 3******************************************************************************/ 4externIDSiDS;/*TheIrregularDataStructure*/ /***************************************************************************** 6/* */ 2 IMPLEMENTATIONNODEINDEXDEFINITION intd2; }; 987structIDSIndex{ 2/* */ intd;/*di(i=,or2)giveapositioninthecorrespondingvector*/ intd;/*anindexfortheirregulardatastructure,avalue */ 3/*functionusedtocreateanewIDSIndex 4IDSIndexidsindex_new(intv,intv,intv2){ 5...allocatetheidsindexandcopyv,v,v2todo,d,d2... 7} 6returniDSIndex; */ 2******************************************************************************/ 2/*functionusedtogivetheindexroot 22staticIDSIndexgetRoot(IGraphiGraph){ 8/***************************************************************************** 9IMPLEMENTATIONGRAPHDEFINITION(IGraph) 24} 23returnidsindex_new(-,-,-)); 25/* */ */ 3/*attheendofavector,wereturnNULLvalue 29/*=>onlyonedimustbedifferentof- 28/*weallowonlymoveinonevectoratatime 27staticIDSIndexnext(intdim,IDSIndexiDSIndex){ 26/*thisfunctionreturnsthenextnodeifexistsindimensiondim */ */ */ switch(dim){ case: if(idsindex->d<ids->v_size-) if((idsindex->d!=-) (idsindex->d2!=-))returnnull; case2: } case:...thesameascasebutfordimensionor2... break; returnidsindex_new(idsindex->d+,idsindex->d,idsindex->d2)); 44staticvoid*getNodeValue(IDSIndexiDSindex){ 4returnNULL; 4} 42/* */ 43/*thisfunctionreturnsforaniDSindexthecorrespondingvalue 45/*onlyonedi(i=,or2)hasavaluedifferentfrom-*/ 46if(iDSIndex->d!=-)return(void*)&(iDS->V[iDSIndex->d]); 47if(iDSIndex->d!=-)return(void*)&(iDS->V[iDSIndex->d]); */ 48if(iDSIndex->d2!=-)return(void*)&(iDS->V2[iDSIndex->d2]); 5} 5/* */ 49returnNULL; Figure5:IGraphdescription 4

16 e f Figure6:Abstractionasadensematrixgraph f 32 void 45 agraph_matrixoperation(agraphagraph, ANodeIndexi,ij; void(*matrixoperation)(inti,intj,void*value)){ 678 for(i=agraph_getroot(agraph);i!=null;i=anodeindex_next(,i)) 9 for(ij=anodeindex_clone(i);ij!=null;ij=anodeindex_next(,ij)) matrixoperation(anodeindex_depth(,ij), 2 } anode_getvalue(ij)); anodeindex_depth(,ij), Figure7:Scanningthedatastructurethroughtheabstractiongraph 5

17 32 void 45 mgraph_matrixoperation(mgraphmgraph, MNodeIndexi; void(*matrixoperation)(inti,intj,void*value)){ 867 for(i=mgraph_getroot(mgraph);i!=null;i=mnodeindex_next(,i)){ 9 matrixoperation(anodeindex_depth(,a), ANodeIndexa=mnodeindex_getANodeIndex(i); 2 } anode_getvalue(a)); anodeindex_depth(,a), Figure8:Scanningthedatastructurethroughthemappinggraph 3 } Figure9:DataDistribution Figure:TraceData 6

18 f f e (a)abstractionasamatrix e f (b)viewasarow Figure:Introductionofaviewgraph 7