A graph based framework for the definition of tools dealing with sparse and irregular distributed data-structures



Similar documents
ibalance-abf: a Smartphone-Based Audio-Biofeedback Balance System

Mobility management and vertical handover decision making in heterogeneous wireless networks

A usage coverage based approach for assessing product family design

QASM: a Q&A Social Media System Based on Social Semantics

Faut-il des cyberarchivistes, et quel doit être leur profil professionnel?

Overview of model-building strategies in population PK/PD analyses: literature survey.

VR4D: An Immersive and Collaborative Experience to Improve the Interior Design Process

Use of tabletop exercise in industrial training disaster.

Study on Cloud Service Mode of Agricultural Information Institutions

FP-Hadoop: Efficient Execution of Parallel Jobs Over Skewed Data

Discussion on the paper Hypotheses testing by convex optimization by A. Goldenschluger, A. Juditsky and A. Nemirovski.

Expanding Renewable Energy by Implementing Demand Response

An update on acoustics designs for HVAC (Engineering)

Online vehicle routing and scheduling with continuous vehicle tracking

Additional mechanisms for rewriting on-the-fly SPARQL queries proxy

Managing Risks at Runtime in VoIP Networks and Services

ANIMATED PHASE PORTRAITS OF NONLINEAR AND CHAOTIC DYNAMICAL SYSTEMS

Information Technology Education in the Sri Lankan School System: Challenges and Perspectives

Minkowski Sum of Polytopes Defined by Their Vertices

Novel Client Booking System in KLCC Twin Tower Bridge

Global Identity Management of Virtual Machines Based on Remote Secure Elements

Aligning subjective tests using a low cost common set

SELECTIVELY ABSORBING COATINGS

Towards Collaborative Learning via Shared Artefacts over the Grid

New implementions of predictive alternate analog/rf test with augmented model redundancy

ANALYSIS OF SNOEK-KOSTER (H) RELAXATION IN IRON

Territorial Intelligence and Innovation for the Socio-Ecological Transition

Application-Aware Protection in DWDM Optical Networks

What Development for Bioenergy in Asia: A Long-term Analysis of the Effects of Policy Instruments using TIAM-FR model

Running an HCI Experiment in Multiple Parallel Universes

Cobi: Communitysourcing Large-Scale Conference Scheduling

Cracks detection by a moving photothermal probe

Towards Unified Tag Data Translation for the Internet of Things

DEM modeling of penetration test in static and dynamic conditions

Virtual plants in high school informatics L-systems

GDS Resource Record: Generalization of the Delegation Signer Model

Performance Evaluation of Encryption Algorithms Key Length Size on Web Browsers

Partial and Dynamic reconfiguration of FPGAs: a top down design methodology for an automatic implementation

An integrated planning-simulation-architecture approach for logistics sharing management: A case study in Northern Thailand and Southern China

Diversity against accidental and deliberate faults

ISO9001 Certification in UK Organisations A comparative study of motivations and impacts.

A modeling approach for locating logistics platforms for fast parcels delivery in urban areas

Optimization results for a generalized coupon collector problem

Florin Paun. To cite this version: HAL Id: halshs

Surgical Tools Recognition and Pupil Segmentation for Cataract Surgical Process Modeling

Multilateral Privacy in Clouds: Requirements for Use in Industry

Advantages and disadvantages of e-learning at the technical university

Improved Method for Parallel AES-GCM Cores Using FPGAs

Heterogeneous PLC-RF networking for LLNs

ControVol: A Framework for Controlled Schema Evolution in NoSQL Application Development

IntroClassJava: A Benchmark of 297 Small and Buggy Java Programs

Methodology of organizational learning in risk management : Development of a collective memory for sanitary alerts

An Automatic Reversible Transformation from Composite to Visitor in Java

The truck scheduling problem at cross-docking terminals

SIMS AND SCANNING ION MICROSCOPY

Distributed network topology reconstruction in presence of anonymous nodes

P2Prec: A Social-Based P2P Recommendation System

Flauncher and DVMS Deploying and Scheduling Thousands of Virtual Machines on Hundreds of Nodes Distributed Geographically

Undulators and wigglers for the new generation of synchrotron sources

Understanding Big Data Spectral Clustering

Hinky: Defending Against Text-based Message Spam on Smartphones

A model driven approach for bridging ILOG Rule Language and RIF

Using Feature Modelling and Automations to Select among Cloud Solutions

Good Practices as a Quality-Oriented Modeling Assistant

Distance education engineering

Business intelligence systems and user s parameters: an application to a documents database

Adaptive Fault Tolerance in Real Time Cloud Computing

Proposal for the configuration of multi-domain network monitoring architecture

Crowdsourcing and digitization

Identifying Objective True/False from Subjective Yes/No Semantic based on OWA and CWA

Comparative optical study and 2 m laser performance of the Tm3+ doped oxyde crystals : Y3Al5O12, YAlO3, Gd3Ga5O12, Y2SiO5, SrY4(SiO4)3O

Donatella Corti, Alberto Portioli-Staudacher. To cite this version: HAL Id: hal

How to use Big Data technologies to optimize operations in Upstream Petroleum Industry

Design and Implementation of Laiwu Black Information Management System Based on ExtJS

Application of computational fluid dynamics to spray drying

Design and architecture of an online system for Vocational Education and Training

Transcription:

A graph based framework for the definition of tools dealing with sparse and irregular distributed data-structures Serge Chaumette, Jean-Michel Lepine, Franck Rubi To cite this version: Serge Chaumette, Jean-Michel Lepine, Franck Rubi. A graph based framework for the definition of tools dealing with sparse and irregular distributed data-structures. 3rd International Workshop on High-Level Programming Models and Supportive Environments (HIPS 98), Mar 998, Orlando, Florida, United States. IEEE Computer Society, pp.62-78. <hal-36323> HAL Id: hal-36323 https://hal.archives-ouvertes.fr/hal-36323 Submitted on 2 Feb 29 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Agraphbasedframeworkforthedenitionoftoolsdealingwith sparseandirregulardistributeddata-structures {Serge.Chaumette,Jean-Michel.Lepine,Franck.Rubi}@labri.u-bordeaux.fr SergeChaumette,Jean-MichelL pineandfranckrubi Tel.:(+335)5684694Fax:(+335)56846669 Lib ration,3345talence,france. LaBRI,LaboratoireBordelaisdeRechercheenInformatique,Universit BordeauxI,35Coursdela Keywords ProgrammingEnvironments,DataParallelism,Tools,Visualization,SparseDataStructures,DistributedData Structures,IrregularDataStructures,Graphs Abstract Industrialapplicationsmainlyusestandarddatastructuressuchasmatrices,butmostofthetimeprovideaspecic asmatricescomingfromthedomainofniteelements[23].thegapbetweentheimplementationandtheabstract cicimplementationsareespeciallyoftenlyusedwhendealingwithlargesparseandirregulardata-structures,such problem-orientedimplementation,e.g.compressedsparsecolumn(csc)seeforinstancesparskit[2].spe- datastructureitimplementsisevenbiggerwhenconsideringdataparallelapplicationswheredatastructuresare distributedoveranetworkofprocessors.hencethereisaneedfortoolsthatmakeitpossiblefordevelopersto visualizeboththeirdata,theirstructure,andtheoperationsthatareappliedtoit,whatevertheireectiveimplementationandtheirdistributionare.suchtoolsmustprovidehighlevelviews,i.e.abstractfromthephysical implementationtoreachthedeveloper'sview.theymustcarrythesemanticsoftheapplicationsandprovide synthesisorlteringmechanismsthatmakeitpossibletofocusonaspecicaspectoftheproblem. prototypestoolsbasedonitwhichwehavedeveloped.theresultingenvironmentiscomposedoftwolayers:the Inthispaper,wepresentaframeworkwhichwehavesetuptosupportthedevelopmentofsuchtools,and rstlayerisamodelthatwehavedenedandimplementedashighlevellibrariesthatmakeitpossibletoeciently ThisworksissupportedbytheFrenchGDR-PRCPRS.

areintegratedwithinagraphicalenvironmentcalledvisit[7]whichispartofthehpfitresearcheort2[7,8,9]. abstractfromtheimplementation;thesecondlayeroersprototypetoolsbuiltontopoftheselibraries.thesetools andgmd/scaiinbonn,germany.theaimofthisprojectistoprovideanintegratedhpfdevelopment HPFITisajointprojectinvolvingthreeresearchlaboratories:LIPinLyon,France,LaBRIinBordeaux,France, environmentthatsupportssparseandirregulardatastructures. Thetwomainresearchdirectionswithinthedataparallelframeworkareexpressionmodes(orlanguages) Introductionandrelatedwork andtools. ulardatastructuresindata-parallellanguages(see[4,5])ortodevelopdedicatedlibraries,toprovide Expressioniswidelystudied.Theaimofmanyinitiativesiseithertointroducesparseandirreg- SPARSEKIT[2],P-SPARSLIB[22]orPETSc[2]). programmerswithreadyimplementationsofpossiblysparsedistributeddatastructures(seeforinstance P2D2[3],Pharos[25],Paradyn[8]).Wewillnotdescribealloftheminthispaperbecausetheyareless Toolsexist,buttheymainlydealwithregular(HPF-like)datastructures(seeforinstanceEPPP[6], relatedtoourgoalwhichistodealwithirregulardatastructures(see[9]foracompletesurvey). theparadigmofdata-parallelprogramming.nevertheless,theresearchwhichhasbeendoneduring Concerningirregularitythereisstillalottobedoneintermsoftoolsthatwouldhelpuserstotackle pastyearsintheareaofmessagepassinghasprovenquitesuccessfulinprovidingsupporttoend-users own.furthermore,publicdomaintools(suchasparagraph[5])arenowbeingdeliveredeitherasfully (seeforinstancetopsys[2,3,6]).bothhardwareandsoftwarevendorssupplyenvironmentsoftheir supportedorsimplyasportedtools.thesetoolsthatwereusedwhendealingwithmessagepassing applicationscanstillbeusedwithintheframeworkofdataparallelismprovidedtheexecutionsupport isdistributed(inwhichcasethecompilertranslatesdataparallelismtomessagepassingparallelism). 2ThisworkispartlysupportedbytheCNRS-INRIAprojectReMaPandbytheFrenchGDR-PRCPRS. 2

Nevertheless,thereisalackofrelationshipbetweentheinformationtheyprovide(whichisintermsof messages)andthesemanticsoftheapplication(whichisintermsofdata).thismakesthemhardlyusable measurement.thereasonforthislackofadequationisthatthebehaviorofanalgorithmismostofthe toexplorethealgorithmicbehavioroftheapplication,althoughtheycanstillbeusefulforperformance timebetterunderstoodwhenconsideringtheabstractstructureitworkswith,ratherthanthephysical implementationofthisdatastructure.forinstanceifthealgorithmhandlesatreethatisimplemented usingavector,itismostprobablythecasethatthisalgorithmcanbebetterunderstoodintermsof requireinformationthatcannotalwaysbeaccessedeasily,suchasdistributionofdata.aconvenientway thetreethanintermsofthevector.oneofthereasonswhyhighleveltoolsaremissingisthatthey implementedinivd[4].thisisalwaysquiteheavyfortheprogrammer.anothermanneristohave toproceedistorelyontheusertosupplytheseinformation.thisistheapproachwhichisforinstance timeasthelanguagelibrariesthemselves.thisistheapproachwhichisachievedinoneofptools librariesthatinstrumentthebasicsofthelanguageandwhicharelinkedtotheapplicationatthesame projectscalleddistributedarrayqueryandvisualization(daqv[7]).thisisnotstraightforwardly portable. data-structuresinsideapplications:wewanttoprovidetoolsdealingwithsuchdatastructuresatahigh Ourapproachisquitedierent.Ouraimisnottoprovidesupportforusingsparseandirregular librariestoabstractfromimplementationanddistributionofdatastructures.theselibrariesareportable levelofabstraction.furthermore,wewanttomakeiteasytodevelopsuchtools.hence,werstdesigned andrelyverylittleontheprogrammer.wethendevelopedtoolsbasedontheselibraries.inthispaper wepresentboththeconceptsoftheselibrariesandthetoolsbasedonthem. weproposeinsection2.section3introducesthewaywemodelizedatastructures.wethendescribethe Therestofthispaperisorganizedasfollows.Werstdescribetheoverallarchitectureofthemodel toolsbasedonourmodel.itisillustratedwithtwosoftwarecomponentswhichwehavedeveloped:data threecurrentlevelsofourmodelinsections4,5and6.section7presentsthegeneralprinciplesofthe DistributionDisplayandTraceDataDisplay.Weeventuallysketchfutureworkdirections. 3

Ourmodeliscomposedoffourlevels.Eachoftheselevels,i.e.implementation,abstraction,mapping 2 Generalarchitectureofthemodel andviewisagraph(gure2):.igraph:theimplementationgraph(gure(a))describestheimplementationofthedatastructure, e.g.threevectorsforacompressedsparsecolumnstorage. intermsofdataitemsandaccessfunctions,i.e.thewaytheyareaccessedwithintheapplication 2.AGraph:theAbstractionGraph(gure(b))describestheabstractdatastructuretheapplication 3.MGraph:theMappingGraphdescribestherelationshipbetweentheIGraphandtheAGraph.This developerhasinminde.g.amatrix. 4.VGraph:theViewGraph(gures(c)and(d))describeshowatoolwilleventuallyseethedata graphisabridgethatcarriesthesemanticsbetweentheimplementationandtheabstraction. Itisnotyetimplemented(seesection8),butonecanworkwiththeAGraph,whichisequivalent structuree.g.acolumnofamatrix.thislevelprovidesforsynthesisandlteringofinformation. Althoughthiswillnotbedetailedhere,themodelprovidesthesameeciencywhenworkingatthe tohavinganidentitybetweentheagraphandthevgraph. mappinggraphlevelorattheimplementationgraphlevel(seegure3). 3Wedeneadatastructureintermsofentrypointsandaccessfunctions.Thisapproachreectstheway Modelizationofadatastructure adatastructureisimplemented.itleadstoamodelizationintermsofagraph.forinstanceavector intv[]canberepresentedbyagraph,therootofwhichisv[]andthenodesarev[i]. AdatastructureDSisasetofdataitemsthatarestructuredbymeansofaccessfunctions. DenitionDatastructure where: DS=(D;d;F;e;E) 4

Successor function for the first vector... Entry Point... e f (a)implementation... Successor function for the third vector f (b)abstractionasamatrix (c)viewasamatrix (d)viewasacolumn disapositiveintegerrepresentingthenumberofaccessfunctions; Disthesetofdataitems; Figure:Architectureoftheframework Fisad-upleofaccessfunctions; eisapositiveintegerrepresentingthenumberofentrypoints; Eisthesetofentrypointsinthedata-structure. canthemselvesbedatastructures,i.e.graphs. Irregulardatastructurescanbedescribedusingthisframework:thenodesofthedatastructuregraph tationtheremightbedierentsetsofaccessfunctions,dependingonhowtheprogrammeraccessesthe Accessfunctionsfiexpresshowoneaccessesthebasicdataitems.Notethatforthesameimplemen- 5

Implementation Graph Mapping Graph Abstraction Graph View Graph Tools (Visualization, traces analysis, debug,...) Figure2:Architectureofthemodel User Efficiency Implementation Mapping Abstraction View data-structure. Figure3:Eciencyofthemodellevels Model Level Denition2Accessfunction Anaccessfunctionisafunctionoverthenodesofthedatastructuresthatbeinggivenanodeproduces anothernode. Eachaccessfunctionmakesitpossibletomovewithinadimensionofthedata-structure. f:n!n n7!f(n) whichrepresentaccessfunctions. Inotherwords,adata-structureisagraph,thenodesofwhichcontainthedataitems,theverticesof Denition3Adatastructureisagraph DS=(N;d;F;r;R) 6

where: Nisasetofnodes; Fisad-upleofsuccessorfunctions; disthenumberofsuccessorfunctions; risthenumberofrootsofthegraph. Risthesetofrootsofthegraph. directaccessdatastructures. Therearesomedatastructures,whereeachdataitemcanbeaccesseddirectly,likesets;wecallthem Denition4Directaccessdata-structure AdatastructureDS=(D;d;F;e;E)issaiddirectaccessifandonlyifE=D,d=,F=;and e=jdj. Insuchacaseallofthedataitemsofthedatastructurearerootsofthegraphusedtodescribeit. likenotationsformulti-dimensionalobjectsrepresentedbygraphs. In[7],weintroducefurtherdenitionsbasedonpreviousworkbyM.Alabau[],thatprovidematrix- Inthissectionwepresentanexample,theaimofwhichistoillustratehowthedenitionsofsection3 4 Describingtheimplementationdatastructure canbeusedtomodelaneectiveimplementation. connectedgraph.thenodesarethevaluesofthethreevectorsplustheentrypoint.wegivethree Theimplementationcontainsthreevectorsofdierentsizes.Anentrypointisaddedtoobtaina successorfunctions;eachfunctionallowstoaccessthenodesofoneofthethreevectors. alsobeinterpretedasacompressedsparserowimplementation,orasanyotherdatastructure.the ThisgraphmaymatchaCompressedSparseColumnimplementationofasparsematrix.Itmay interpretationofthisimplementationasagraphdoesnotdescribethesemanticsofwhatiseectively implemented.thebenetisthatthedescriptionofthisgraphbytheprogrammerisstraightforward. 7

Successor function for the first vector... Entry Point...... Figure5showsasampleofthecodetheprogrammermustwriteinordertodescribethisimplementation. Figure4:Agraphmodelizationoftheirregularstructure Successor function for third vector 5Wecallabstractiontheabstractdatastructurethattheuserwantstomanipulate.Sinceanabstraction Describingtheabstractiondata-structure aimoftheabstractionistooeraninterpretationoftheimplementation.forinstancethethreevectors isitselfadata-structure,itcanberepresentedbyanentrypointandaccessfunctions,i.e.agraph.the ofsection4canbeinterpretedasacscstorageofamatrix(seegure6). abstractiongraph. Hence,abstractingfromanimplementationconsistsinmappingtheimplementationgraphtothe 6Themappingestablishesacorrespondencebetweentheimplementationgraphandtheabstractiongraph. Mapping Itisusedtomovefromtheabstractiontotheimplementationandvice-versa. graphandanodeoftheabstractiongraph.suchanodecanonlyexistprovidedthereisanodeofthe Itisagraph,thenodesofwhicharedenedasapaircontainingbothanodeoftheimplementation implementationcorrespondingtothegivennodeoftheabstraction(theabstractionwhichisdenseusually hasmorenodesthantheimplementationwhichissparse).assumewehaveafunctionthatgivenanode 8

datastructure).nowonwewillcallthisfunctionabs_to_imp. oftheabstractionreturnseitheranodeoftheimplementationornull(probablyaholeinthesparse Denition5Nodes=f(abstoimp(anode);anode)=anode2agraphenodesg Wecanthendenethenodesofthemappinggraphasfollows: RemarkWecanalsodeneafunctionimp_to_abs(implementationnodetoabstractionnode).It allowsanothersymmetricdenitionofthemappinggraph.theusercandenethetwofunctions(togain eciency).oneofthesefunctionsmaybediculttowrite:itcanbelefttothetoolstoconstructit usingtheotherfunction,storingthisinformationinahashtable. 7Theframeworkpresentedaboveisbeingusedtodevelophighleveltools.Inthissectionwepresenttwo Usage prototypes. 7. Usingourlibraries,highleveltoolsworkonlywithgraphs,whichtheycanforinstancescan,usingeither Generalprinciple oftheloopsshowngures7and8. graphs.thisisusedforinstancebythetracesvisualizationtooldescribedbelow. Furthermore,thelibrarymakesitpossibletoattachadditionalinformationtoanynodeofanyofthe Thegoalofourtoolsistooerasetofsoftwarecomponentstovisualizeandanalyzethebehaviorof 7.2 Eectiveusage data-parallelprogramsintermsofthedatatheyworkwith.theframeworkpresentedaboveisbeingused viewgraphs. todevelopthesetools.theyareimplementedeitherintermsofabstractiongraphs,mappinggraphsor DDD(DataDistributionDisplay)andTDD(TraceDataDisplay).Partsofthesetoolsarecomponents Asofwriting,someofthesetoolsareavailableasprototypes.Thissectiondescribestwoofthem: 9

ofthehpfitproject.acompletedescriptioncanbefoundin[2].weonlypresentscreendumpshere. TheycomefromaCholeskyfactorizationofasparsesymmetricpositivedenitematrix.Theparallel machinewhichhasbeenusedtorunittocollecttracesisanibmsp2. languageandcompiledusingadaptor[]extentedwithalibrarycalleddddt(distributedderived Weinterfacedourtoolswithadataparallelsystem.ThesourceprogramiswritteninaHPF2-like Foranecientparallelexecution,aspecicirregulardistribution[]isused. DatatypeforTree).Thislibraryallowsthemanipulationofhierarchicalaccessirregulardatastructures[7]. 7.3 Figure9showsthedistributionofthedataonthevirtualprocessors(onecolorbyprocessor). DataDistributionDisplay dataitem(resp.processor).datadistributiondisplaycanbeusedbeforetheexecutionifthedistribution Usingthemouse,theusermayselectavirtualprocessor(resp.data)andvisualizethecorresponding isregularorcomputable. Theparallelapplicationisinstrumentedtogeneratetracesintermsofaccessestodataitems.Atruntime, 7.4 TraceDataDisplay TraceDataDisplayisapostmortemtoolbasedonruntimegeneratedtracesandonthemappinggraph atracecollectorrecordseventsoneachprocessor.filteringmethodsareusedtolimittheamountoftraces. ofourmodel. thecholeskyfactorization[].thiscanforinstancehelptheusertoestimatethequalityofthedata Figureshowswhichdataitemsareinvolvedinremotereadoperationsduringtherstpartof distribution. Inthispaperwehavesetupaformalapproachtothemodelizationofsparseandirregulardatastructures. 8 Conclusionandfuturework Usingthisframework,wehaveshownthatitispossibletodescribeimplementations,onlyintroducing semanticsinthemappingfromtheimplementationtotheabstraction.dependingofwhattheyaredoing,

toolsbuildontopofthisframeworkcaneitherusetheabstractiongraph,or,tobemoreecient,usethe mappinggraphthatmakesitpossibletoscanthedatastructureavoidingtolookatholes. ourmodel,weareprovidedwithanabstractionthatrepresentsamatrixasshowngure(a).theuser Anotherlevelofabstractionthatwestillhavetoimplementisthatofltering.Assumethatwithin maywanttoseeeitherthematrixitself,or,forinstance,agivenrowofthismatrix(seegure(b)), orevenallitemsofeachrowasauniqueitem,i.e.heneedstoltertheabstraction.thereforeweare workingonthedenitionofaviewgraphwithamappingfromtheabstractiontoit..designinglibrariesthatwouldprovidestandardabstractionsusingthismodelforstandardimplementations,likethoseofsparskitforinstance; Thenextstepswithinthisprojectare: 2.extendingthehighleveltoolsbasedonthismodel,toaddthemtoVisit[7]insidetheHPFIT 3.validatingthetoolswiththeend-userstoproposeviewsadaptedtotheirneeds; environment; 4.interfacingthisframeworktoexistingsoftwareenvironmentssuchasDAQV[7]orEMILY[24]. References []M.Alabau.Uneexpressiondesalgorithmesmassivementparall les structuresdedonn esirr guli res.th se, [2]T.Bemmerl.Anintegratedandportabletoolenvironmentforparallelcomputers.InProceedingsoftheIEEE LaBRIUniversit BORDEAUXI,September994. [3]T.BemmerlandA.Bode.Anintegratedenvironmentforprogrammingdistributedmemorymultiprocessors.InBodeA.,editor,ProceedingsoftheSecondEuropeanDistributedMemoryComputingConference InternationalConferenceonParallelProcessing(St.Charles,USA),pages553,988. [4]A.J.C.BikandH.A.G.Wijsho.Compilationtechniquesforsparsematrixcomputations.Technical (M nchen),volume487oflecturenotesincomput.sci.,pages342.springer-verlag,99. report,universityofleiden.

[5]A.J.C.BikandH.A.G.Wijsho.Advancedcompileroptimizationsforsparsecomputations.Journalof [6]A.Bode.Developmentsindistributedmemoryarchitectures.InProceedingsofMicrosystem'9(Bratislava, ParallelandDistributedComputing,(3):424,995. Systems,TUM-I93,SFB-BerichtNr.342/9/9A,January99,seiten6. MethodenundWerkzeugef rdienutzungparallelerrechnerarchitekturen,topsys,toolsforparallel CSSR),99.AlsoinTechnischeUniversit tm nchen,institutf rinformatik,sonderforschungsbereich342: [7]T.Brandes,S.Chaumette,M.-C.Counilh,A.Darte,J.C.Mignot,F.Desprez,andJ.Roman.HPFIT:ASet ofintegratedtoolsfortheparallelizationofapplicationsusinghighperformancefortran:partii:data StructuresVisualizationandHPFSupportforIrregularDataStructureswithHierarchicalScheme.Parallel [8]T.Brandes,S.Chaumette,M.-C.Counilh,A.Darte,J.C.Mignot,F.Desprez,andJ.Roman.HPFIT:ASet Computing,996.EditedbyJ.DongarraandB.Tourancheau. andthetranstoolenvironment.parallelcomputing,996.editedbyj.dongarraandb.tourancheau. ofintegratedtoolsfortheparallelizationofapplicationsusinghighperformancefortran:parti:hpfit [9]T.Brandes,S.Chaumette,andF.Desprez.TransTOOL:atoolforportingscienticapplicationsonparalleldistributedmemorymachines.In2ndEuropeanSchoolofComputerScience,ParallelProgramming []T.BrandesandF.Zimmermann.ADAPTOR-ATransformationToolforHPFPrograms.InK.M.Decker EnvironmentsForHighPerformanceComputing,996. andr.m.rehmann,editors,programmingenvironmentsformassivelyparalleldistributedsystems,pages []P.Charrier,FackL.,andJ.Roman.Blockdatapartitionforparallelnesteddissection.InProceedingsofthe 996.Birkh userverlag,april994. [2]S.Chaumette,F.Rubi,andLepineJ.M.Internalreport,LaBRI,Universit Bordeaux-I,997.Tobe 7thSIAMconferenceonparallelprocessingforscienticcomputing.SiamEditions,995. [3]DoreenChengandRobertHood.Aportabledebuggerforparallelanddistributedprograms.InProc.of published. [4]M.C.Hao,A.H.Karp,M.Mackey,V.Singh,andJ.Chien.On-the-yvisualizationanddebuggingofparallel Supercomputing'94,994. programs. 2

[5]M.T.HeathandJ.A.Etheridge.Visualizingtheperformanceofparallelprograms.IEEESoftware,8(5):2939, [6]G.Hurteau,V.VanDongen,andG.Gao.OverviewofEPPP-anEnvironmentforPortableParallelProgramming.InProceedingsofSupercomputingSymposium'94,Canada'sEighthAnnualHigh ftp://ftp.crim.ca/apar/public/papers/994/ss94-eppp.ps.gz. PerformanceComputingConference,pages927,Toronto,Ontario,June994.Alsoavailableas September99. [7]A.Malony,MaryZosel,MayJohn,AlanKarp,andDavidPresberg.PTOOLSprojectproposalDistributedArrayQueryandVisualization.Availableashttp://www.cs.uoregon.edu/hacks/research/ptoolsdaqv/proposal. [8]BartonP.Miller,MarkD.Callaghan,JonathanM.Cargille,JereyK.Hollingsworth,R.BruceIrvin, toolsforparallelanddistributedcomputersystems. KarenL.Karavanic,KrishnaKunchithapadam,andTiaNewhall.TheParadynParallelPerformanceMeasurementTools.IEEEComputer,28():674,November995.Specialissueonperformanceevaluation [9]J.LPazat.SpringSchoolondataParallelism,chapterToolsforHighPerformanceFortran:aSurvey.Springer [2]BalayS.,GroppDW.D,McInnesL.C.,andSmithB.F.ModernSoftwareToolsinScienticComputing, Verlag,996. chapterecientmanagementofparallelisminobject-orientednumericalsoftwarelibraries,pages6322. [2]Y.Saad.SPARSEKIT:ABasicToolsKitforSparseMatrixComputations,June994.Version2.Documentation. BirkhauserPress,997.Alsoavailableathttp://www.mcs.anl.gov/petsc/petsc.html. [22]Y.SaadandA.V.Malevsky.P-SPARSLIB:aportablelibraryofdistributedmemorysparseiterativesolvers, [23]G.StrangandG.F.Fix.AnAnalysisoftheFiniteElementMethod.PrenticeHall,973. 995.Documentation. [24]LoosT.andBramleyR. http://www.cs.indiana.edu/scicomp/emily.html. EMILY:avisualizationtoolforlargesparsematrices. Availableas [25]ThePHAROSTeam.ThePHAROSproject.Availableashttp://www.vcpc.univie.ac.at/activities/projects/PHAROS/. 3

5/*composedhereof3Vectorsofdifferentsizes*/ 3******************************************************************************/ 4externIDSiDS;/*TheIrregularDataStructure*/ /***************************************************************************** 6/*---------------------------------------------------------------------------*/ 2 IMPLEMENTATIONNODEINDEXDEFINITION intd2; }; 987structIDSIndex{ 2/*---------------------------------------------------------------------------*/ intd;/*di(i=,or2)giveapositioninthecorrespondingvector*/ intd;/*anindexfortheirregulardatastructure,avalue */ 3/*functionusedtocreateanewIDSIndex 4IDSIndexidsindex_new(intv,intv,intv2){ 5...allocatetheidsindexandcopyv,v,v2todo,d,d2... 7} 6returniDSIndex; */ 2******************************************************************************/ 2/*functionusedtogivetheindexroot 22staticIDSIndexgetRoot(IGraphiGraph){ 8/***************************************************************************** 9IMPLEMENTATIONGRAPHDEFINITION(IGraph) 24} 23returnidsindex_new(-,-,-)); 25/*---------------------------------------------------------------------------*/ */ 3/*attheendofavector,wereturnNULLvalue 29/*=>onlyonedimustbedifferentof- 28/*weallowonlymoveinonevectoratatime 27staticIDSIndexnext(intdim,IDSIndexiDSIndex){ 26/*thisfunctionreturnsthenextnodeifexistsindimensiondim */ */ */ 35 34 3switch(dim){ 32 33 case: if(idsindex->d<ids->v_size-) if((idsindex->d!=-) (idsindex->d2!=-))returnnull; 38 39 36 37 case2: } case:...thesameascasebutfordimensionor2... break; returnidsindex_new(idsindex->d+,idsindex->d,idsindex->d2)); 44staticvoid*getNodeValue(IDSIndexiDSindex){ 4returnNULL; 4} 42/*---------------------------------------------------------------------------*/ 43/*thisfunctionreturnsforaniDSindexthecorrespondingvalue 45/*onlyonedi(i=,or2)hasavaluedifferentfrom-*/ 46if(iDSIndex->d!=-)return(void*)&(iDS->V[iDSIndex->d]); 47if(iDSIndex->d!=-)return(void*)&(iDS->V[iDSIndex->d]); */ 48if(iDSIndex->d2!=-)return(void*)&(iDS->V2[iDSIndex->d2]); 5} 5/*---------------------------------------------------------------------------*/ 49returnNULL; Figure5:IGraphdescription 4

e f Figure6:Abstractionasadensematrixgraph f 32 void 45 agraph_matrixoperation(agraphagraph, ANodeIndexi,ij; void(*matrixoperation)(inti,intj,void*value)){ 678 for(i=agraph_getroot(agraph);i!=null;i=anodeindex_next(,i)) 9 for(ij=anodeindex_clone(i);ij!=null;ij=anodeindex_next(,ij)) matrixoperation(anodeindex_depth(,ij), 2 } anode_getvalue(ij)); anodeindex_depth(,ij), Figure7:Scanningthedatastructurethroughtheabstractiongraph 5

32 void 45 mgraph_matrixoperation(mgraphmgraph, MNodeIndexi; void(*matrixoperation)(inti,intj,void*value)){ 867 for(i=mgraph_getroot(mgraph);i!=null;i=mnodeindex_next(,i)){ 9 matrixoperation(anodeindex_depth(,a), ANodeIndexa=mnodeindex_getANodeIndex(i); 2 } anode_getvalue(a)); anodeindex_depth(,a), Figure8:Scanningthedatastructurethroughthemappinggraph 3 } Figure9:DataDistribution Figure:TraceData 6

f f e (a)abstractionasamatrix e f (b)viewasarow Figure:Introductionofaviewgraph 7