issuitableforexecutiononasynchronous,tightly-coupledparallelmachine,suchasasuper-scalaror 1Introduction

Transcription

1 UniversityofCalifornia,Berkeley ComputerScienceDivision Berkeley,CA AlexanderAiken Resource-ConstrainedSoftwarePipelining DepartmentofInformationandComputerScience StevenNovacky DepartmentofInformationandComputerScience UniversityofCalifornia,Irvine AlexandruNicolau UniversityofCalifornia,Irvine Irvine,CA92717 Irvine,CA92717 experimentswithanimplementationarealsopresented. schedulingchoicesaremadewithtrulyglobalinformation.proofsofcorrectnessandtheresultsof generalityinthesoftwarepipeliningalgorithmisnotsacricedtohandleresourceconstraints,and smoothlyintegratesthemanagementofresourceconstraintswithsoftwarepipelining.furthermore, parallelismingeneralloops.thealgorithmaccountsformachineresourceconstraintsinawaythat Thispaperpresentsasoftwarepipeliningalgorithmfortheautomaticextractionofne-grain Abstract issuitableforexecutiononasynchronous,tightly-coupledparallelmachine,suchasasuper-scalaror 1Introduction VLIW(VeryLongInstructionWord)machine. Recentlytherehasbeenconsiderableinterestinaclassofcompilerparallelizationtechniquesknown overlappingtheoperationsofaloopbodyinmuchthesamewaythatahardwarepipelineoverlaps operationsinadynamicinstructionstream.theschedulecomputedbyasoftwarepipeliningalgorithm collectivelyassoftwarepipelining.softwarepipeliningalgorithmscomputeastaticparallelschedule tectures.thesecondreasonisthatthesetightly-coupledmachinesmustbeprogrammedataverylow computermanufacturers e.g.,hp,phillips,siemens arealsodevelopingvliworsuper-scalararchi- synchronousmachinebuilttodateismultiow'strace-14,whichhas14functionalunits.several super-scalarandvliwmachinesarebeingbuilt.ibm'ssystem6000canexecutefouroperationsin parallel;intel'si860andi960chipscanexecutethreeoperationsinparallel.thelargesttightly-coupled Softwarepipeliningalgorithmsareinterestingforatleastthreereasons.Therstreasonisthat ThisworksupportedinpartbyONRgrant#N K0215 ythisworksupportedinpartbyonrgrant#n k0215 1

2 intogoodparallelschedules. tiontimingsandresourceconictsbetweenfunctionalunits.thistaskisextremelytime-consumingand error-prone;compilationtechniquesareneededtotranslateprogramswrittenatareasonablyhighlevel meansthatpersonmustknowaboutandaccountfordetailsofthehardwaredesignsuchasinstruc- level.someonewritingaprogramforatightly-coupledmachinemustdevelopaparallelschedule,which fastercompilationtimethanotherschedulingtechniques.thispotentialisillustratedbytheexample infigure1.figure1ashowsasimplesequentialloopandfigures1band1cshowtwodierentparallel schedulesfortheloop.forconvenience,welabeltheoperationsintheoriginalloopa{dandreferonly totheselabelsintheparallelloops.inthisexample,someparallelismispresentwithintheloopbody (becauseoperationsbandccanbeexecutedsimultaneously)aswellasacrossiterations(becausedfrom Thenalreasonisthatsoftwarepipeliningtechniquesholdthepromiseofproducingbettercodewith oneiterationcanoverlapwithafromthenextiteration).theclassicalapproachtoschedulingtheloop infigure1aistounrolltheloopbodysomenumberoftimesandthenapplyschedulingheuristicswithin insideandacrossalliterationsofaloop;hencesoftwarepipeliningachievestheeectofschedulingwith impossibleorimpracticaltoobtain.softwarepipeliningprovidesadirectwayofexploitingparallelism bothinsideandacrossiterationscouldbeexploitedbythisapproach.however,fullunrollingisusually theunrolledloopbody[fis81]asillustratedinfigure1b.whilethisapproachallowsparallelismto fullunrolling.asoftware-pipelinedversionoftheoriginalloopisgiveninfigure1c. beexploitedbetweensomeiterationsoftheoriginalloop,thereisstillsequentialityimposedbetween iterationsoftheunrolledloopbody.ingeneral,iftheloopcouldbefullyunrolled,allparallelism includeasoftwarepipeliningalgorithmthatgeneratesoptimalcodeforloopswithoutconditionaltests 1.1PreviousWork Onebodyofworkonsoftwarepipelininghasfocussedonestablishingtheformalismrequiredtoadequately addresswhatsoftwarepipeliningalgorithmscanandcannotachieve.resultsinthislineofdevelopment [AN88a]andaproofthatoptimalsoftwarepipeliningisimpossibleingeneral[SGE91].However,this workhaslargelyignoredresourceconstraints. rithmsthetreatmentofresourceconstraintsisintimatelyconnectedtosoftwarepipelining thatis,the tersoftwarepipelining[npa88].severalsoftwarepipeliningalgorithmsaccountforresourceconstraints directlyaspartofthesoftwarepipeliningalgorithm,e.g.[rg81,lam87].however,inmostsuchalgorithmsdealwithonlyweakformsofresourceconstraints e.g.,thenumberofoperationsthatcanbe executedinparallel.othersassumeresourceconstraintsarehandledinaseparate\x-upphase"af- Existingsoftwarepipeliningalgorithmshandleresourceconstraintsinavarietyofways.Somealgo- softwarepipeliningisnotseparablefromthehandlingofresourceconstraints.oneofourinterestsisto separatewhatisreallyintrinsictosoftwarepipeliningfromother,orthogonalconcerns.amoreextensive discussionofpreviousandrelatedworkisincludedinsection9. 2

3 -a:i b:j c:k i+1 d:l? i+h - i+g 1:b,c 0:a? (a)anexampleloop. j+1 2:d,a 3:b,c 4:d (b)unrolledtwiceandscheduled.? -2:d,a 1:b,c 0:a? Figure1:LoopUnrollingandSoftwarePipelining (c)pipelinedloop.? 3

4 1.2OurApproach Inthispaper,wepresentanalgorithmthatsmoothlyintegratessoftwarepipeliningwiththetreatment ofresourceconstraints,whileatthesametimemaintainingastructureddesignthatseparatesorthogonal concerns.ouralgorithmservestwopurposes.first,webelievethealgorithmrepresentsapractical directionandcanformthebasisofimplementationsofsoftwarepipelining;wediscussanimplementation ofouralgorithminsection7.second,thealgorithmrepresentsasummaryofmanyofthemostinteresting aspectsofourinvestigationofsoftwarepipeliningoverthelastseveralyears[nic85,an88a,an88b,aik88, Aik90,AN91].Ouralgorithmhasseveralnovelfeatures: Thehandlingofresourceconstraintsisorthogonaltothesoftwarepipelining. Ateachstepthealgorithmhasglobalinformationabouttheoperationsthatcanbescheduled. allocationdecisions(seesection8)andnotbythedesignofthesoftwarepipeliningalgorithm. machine)andnochangeswouldberequiredintheoverallalgorithm.thesecondandthirdpointstogether implythatthequalityofthenalpipelinedloopislimitedonlybytheabilitytomakegoodresource InatechnicalsensedenedpreciselyinSection8,givensucientresourcesouralgorithmcan Theadvantageoftherstpointisthatthetreatmentofresourcescouldbemodied(say,foradierent Oursoftwarepipeliningalgorithmisbuiltfromtwocomponents:aschedulerandadependencean- producecodearbitrarilyclosetothetheoreticaloptimum. alyzer.themachine-dependentschedulerisusedtoincrementallybuildaparallelizedloopfromase- quentialloop.foreachparallelinstruction,theschedulerselectsoperationstoschedulebasedonthe constructed,thesoftwarepipeliningalgorithmchecksforrepeatingstatesthatcanbe\pipelined."the setofoperationsavailableforschedulinginthatinstructionandavailableresources.thesetofavailable onthescheduleranddependenceanalyzerthatguaranteethecorrectnessandterminationofthesoftware toplaceoperations,thesetofavailableoperationsisupdatedincrementally.together,thescheduler andthedependenceanalyzerencapsulateallmachine-dependentinformation.astheparallelizedloopis softwarepipeliningalgorithmitselfisverysimple;thedicultyliesinestablishingminimalrestrictions operationsismaintainedbyaglobaldependenceanalyzer;astheschedulermakesdecisionsaboutwhere correctness.section5givesanalgorithmforincrementallymaintainingthesetofavailableoperations. usedtodevelopthealgorithm.section3worksthroughasmallexampletogiveanintuitiveideaofhow thesoftwarepipeliningalgorithmworks.section4describesthealgorithmandpresentsaproofof Section6describestheintegrationofresourceconstraintsintothealgorithm.Handlingresourceswellis pipeliningalgorithm. criticalinrealisticapplicationsofsoftwarepipelining.section7brieydescribesanimplementationofour Therestofthispaperisdividedintoninesections.Section2denesthemodelofparallelcomputation algorithm,someadditionaloptimizations,andsomeexperimentalresults.theexperimentalresultsbear outthestrengthsofourapproachandpointoutsomeweaknesses;botharediscussedatlength.section8 presentsaresultthatsuggestsouralgorithmcanachievethebestschedulespossibleinthepresenceof 4

5 resourceconstraints.adiscussionofrelatedworkisinsection9.thenalsectionsummarizesand presentssomeconclusions. 2BasicTerminology Thissectiondevelopsasimplemodelofatightly-coupled,synchronousparallelmachine.Theformalism dividedintoassignmentsthatreadandwriteaglobalstore,tests(boolean-valuedfunctions)thataect isusedtoexplainoursoftwarepipeliningalgorithmandtoprovideabasisforaproofofcorrectness. theowofcontrol,andadistinguishedoperationstop. program.associatedwitheachstatenisops(n),theoperationsofn,whichareelementsofx.the statesrepresentparallelinstructions;intuitively,whencontrolreachesastaten,alloperationsinops(n) AprogramisanautomatonhX;;n0;Ni.Xisasetofnoperationsfx0;:::;xn 1g.Operationsare areexecutedsimultaneously.tosimplifythepresentation,weassumethatalloperationsexecuteinunit time.extensionstomulti-cycleoperationsandpipelinedfunctionalunitsarediscussedinsection6. ThebodyoftheprogramisasetNofstatesn0;:::;nm 1.Thestaten0isthestartstateofthe registers).thetransitionfunctionmapscongurationsintocongurations.anexecutionisasequence ofcongurationsh:::;hni;sii;:::isuchthat(hni;sii)=hni+1;si+1i. Acongurationisapairhn;siwherenisastateandsisastore(thecontentsofmemorylocationsand organizedasabinarydecisiontreewithauniqueroot.onebranchofeachtestinthedecisiontreeis parallel(multi-wayjumps).asanexample,inonepossiblemodeltestswithinastatenarealways functionsofsuper-scalarandvliwmachinesarecomplexandvaryconsiderablyfrommachinetomachine.thegreatestsourceofcomplexityisdeningwhatitmeanstoexecutemorethanonetestin parallelinstruction.wedeliberatelyavoiddeningatransitionfunctioninanydetail.thetransition Thetransitionfunctiondescribeshowatightly-coupled,synchronousmachineactuallyexecutesa labeledtrue,theotherislabeledfalse.eachleafofthedecisiontreeisapointertoanotherstate.when thestateisexecuted,allofthetestsareevaluatedinparallelinthestore.thenextstatetobeexecuted isdeterminedentirelybytests;thatis,theresultofevaluatingthetestsinastatedeterminesthenext havebeenproposedandimplemented[fis80,kn85,aag+86,ebc87].thesoftwarepipeliningalgorithm wepresentappliestoanyofthesecontrol-owmechanisms. istheleafthatterminatesthe(unique)pathfromtherootwhereeverybranchislabeledbythevalueof thattestinthestore.thereareotherpossibleimplementationsofmulti-wayjumps;manymechanisms state.abranchofastatenisatruthassignmenthx1=true;x2=false;:::itothetestsx1;x2;:::in consistingoftheemptytruthassignment.thefunctionsucc-on-branchmapsastatenandabranch c2branch(n)toasuccessornoden0(thenamesucc-on-branchstandsfor\successoronbranch").1we n.thesetofallbranchesofnisbranch(n);ifnhasnotests,thenbranch(n)isthesingletonsetfhig Weusethefollowingabstractionofcontrol-owthroughoutthispaper.Weassumethatcontrol-ow branchesseparatelyinouralgorithm;inanimplementationforaparticularcontrol-owmechanismmanybranchescanbe 1Notethatinmostcasesanodewithktestswillnothave2kdistinctsuccessors.Forgenerality,wetreateachofthe2k 5

6 evaluationoftestsincongurationhn;sisatisesthetruthassignmentc. assumethatifsucc-on-branch(n;c)=n0,thenthereisastatessuchthat(hn;si)=hn0;s0iandthatthe Denition2.1LetPbeaprogramhX;;n0;Ni.Ifthereisanexecutionhhn0;si;:::;hnk;s0iisuchthat pipeliningalgorithmiscorrect(i.e.thatitpreservesthemeaningoftheoriginalprogram). operationsandcannothaveanysuccessors. controlistransferredtosomen02succ(n).astatethatcontainstheoperationstopcannotcontainother Wenextdeneameaningfunctionforprograms,whichisusedintheproofthatoursoftware Thesetofsuccessorssucc(n)ofastatenisfn0j9cs:t:succ-on-branch(n;c)=n0g.Whennisexecuted, operationsx0;:::;xi 1andistatesn0;:::;ni 1wherenj=fxjg.Allbackedgesgotothestartstaten0; thatis,ifni2succ(nj)andijtheni=0.everystateisassumedtobereachablefromthestartstate. inparallelizing.forconvenience,weusethefollowingdenition.asequentialloopisaprogramwithi ops(nk)=fstopgthen(p;s)=s0.ifnosuchexecutionexists,then(p;s)=?. Softwarepipeliningisaloopparallelizationtechnique,sowemustdescribetheloopsweareinterested ProgramsP1andP2areequivalent(P1P2)if8s(P1;s)=(P2;s). schedulingasubsetoftheavailableoperationsasthestartstate,thealgorithmrecursivelyschedulesthe successorsofthestartstatebyconsideringwhatoperationscanbescheduledinthesuccessorstates,and fromthesequentiallooplthatlegallycanbescheduledasthestartstateoftheparallelloop.after L.Initially,theparallelizedloopisempty(hasnostates)andthealgorithmchoosesasetofoperations 3AnExample soon.themaindicultyisguaranteeingthatthisprocedureterminates.weshowthateventuallythe GivenasequentialloopL,oursoftwarepipeliningalgorithmincrementallybuildsaparallelizedloopfrom beingconstructed.howthissetisbuiltandmaintainedisdiscussedinsection5.fornowitisonly operations.ateachstep,acontainsasetofoperationsavailableforschedulinginthecurrentstate scheduledstatesmustfallintoadetectablerepeatingpattern,atwhichpointaloopcanbeconstructed statewithoutviolatingprogramsemantics. importanttounderstandthatsetacontainsalloperationsthatcouldbescheduledlegallyinthecurrent fromthispatternofrepeatingstates. Initially,thenewprogramgraphisemptyandAcontainsalloperationsavailableforschedulingin AnimportantdatastructureusedbythealgorithmisanincrementallymaintainedsetAofavailable therststate.considertheprograminfigure2.wedisplayprogramsascontrol-owgraphswiththe conventionthattruebranchesoftestsaretotheleftandfalsebranchesaretotheright.notalloperations canbescheduledintherststate;forexample,cmustbescheduledafterb,sincecreferencesavalue thatbwrites.instandardcompilerterminology,thereisadatadependencefrombtoc[kkp+81]. executionofastate,andwriteconictsarenotpermitted.inthismodel,theoperationsa,b,andfare treatedtogether. Forthisexample,weassumeamachinemodelinwhichallreadstakeplacebeforeanywritesduring 6

7 -a:a[i] b:j f(a[i 1]) d:b[j] A[j] c:ifa[j]<0? t@@@r fg:stop e:b[j] A[j] allavailableforschedulingintherststate.becausethealgorithmmayoverlapoperationsfromdierent iterations,wesuperscriptoperationswiththeschedulediterationfromwhichtheycame.inaddition, wesubscriptavailableoperationsetstokeeptrackofdierentvaluesfordierentstates.thus,initially A0=fa0;b0;f0g. Figure2:Anexampleloop. bytheintegeri.therestofthissectiondescribeshowthesoftwarepipeliningalgorithmcomputesthis ofoperationstoscheduleinthecurrentstate.together,theproceduretomaintainthesetofavailable algorithmitselfisbuiltontopofthesetwocomponents. operationsandtheschedulerencapsulateallmachine-dependentinformation.thesoftwarepipelining ApipelinedversionoftheloopinFigure2isgiveninFigure3.InFigure3,thestateniislabeled Anothercomponentofthepipeliningalgorithmisthescheduler.TheschedulerselectsfromAaset betwosuccessorsoftherststate oneforthecasewherefevaluatestotrue,andoneforthecasewhere parallelschedulefromthesequentialloop.fortherststaten0,assumingthatthemachinehassucient factthata0,b0andf0havebeenscheduledandthatthisbranchoff0istheloopexit.becausewrite resources,theschedulercouldchoosetoscheduleallavailableoperations.becausefisatest,therewill conictsarenotpermitted,d0ande0cannotbescheduledinthesamestate,butbothare\available" at fevaluatestofalse.thesetsofavailableoperationsaredierentforthetwosuccessors. programterminatesonthisbranch.thenewseta1ofavailableoperationsisfc0;d0;e0g,reectingthe Considerthesuccessorn1ofn0forthecasewheref0evaluatestofalse.Thiscaseiseasy,asthe 7

8 5:b,f0:a,b,f t-f 4:c,d,a?HHHHHj 6:e,b,f tf6 1:c,d 3:g?AAAU 2:e thispoint,alldependencesonthetwostatementshavebeensatised.assumethattheschedulerselects operationsc0andd0forstaten1.operationc0isatest,sotherearetwosuccessorsofthisstate.for thesuccessorn2wherec0evaluatestofalse,thesetofavailableoperationsa2isfe0g.assumethatthe schedulerplacese0inn2.forthesinglesuccessorn3ofn2thesetofavailableoperationsa3isjustfg0g, Figure3:Theloopaftersoftwarepipelining. isavailableforschedulinginparallelwithstatementsfromtherstiteration.asubtlepointisthat thestopoperation.thusn3containsonlyg0.backingupton1,thesetofavailableoperationsforthe newsetofavailableoperationsa4isfc0;d0;e0;a1g.notethattheoperationa1fromtheseconditeration branchwherec0evaluatestotrueisalsofg0g,sothesuccessorofn1onthispathisalson3. because(asbefore)d0ande0cannotbescheduledinthesamestate.eventhoughalloperationsthat operationb1isnotavailableforscheduling,eventhoughallreadstakeplacebeforeallwritesandall operationsfromtherstiterationthatreadvariablejareavailableina4.operationb1isnotavailable Thiscompletestheterminatingpathfromn0.Ontheotherpath,wheref0evaluatestotrue,the frombeingavailable. readjareina4,notallofthesecanbescheduledinn4,andthisfactpreventsstatementsthatwritej thissetisexactlythesameasa4.thesuperscriptsarejustawayofkeepingtrackoftheiterationof thesuccessorofn5onthepathwheref1istrueisfc1;d1;e1;a2g.notethat,exceptforthesuperscripts, eachoperation;thesetshavethesameoperations.ratherthancontinueschedulingatthispoint,the fb1;f1g.assumingthattheschedulerplacesbothoperationsinn2,thesetofavailableoperationsfor therearetwosuccessorsofthisstate.forthesuccessorn5wherec0evaluatestotrue,theseta5is Assumethattheschedulerselectsoperationsc0,d0,anda1forstaten4.Operationc0isatest,so successorofn5wheref1evaluatestofalseisfc1;d1;e1g.exceptforsuperscripts,thisisexactlythesame asa1.asbefore,thepipeliningalgorithmmakesn1asuccessorofn5. pipeliningalgorithmsimplymakesn4asuccessorofn5.similarly,thesetofavailableoperationsforthe 8

9 c:a[i+1] - b:a[i] a:j 2 (idiv50) A[i j]+k A[i+1 j]+h d:i++ (a)anotherexampleloop.? -1:b,c 0:a? Figure4:Anotherexampleloop. (b)anincorrectschedule. 2:d? ThesetofavailableoperationsA6isfe0;b1;f1g.Assumingthattheschedulerplacesallthreeoperations proceedsjustasitdidforn5.thealgorithmterminateswiththescheduleinfigure3. inn6,thesetsofavailableoperationsforthetwosuccessorsofn6arethesameasforn5andscheduling Backingup,thepipeliningalgorithmnextconsidersthesuccessorn6ofn4wherec0evaluatestofalse. becausetwosetsofavailableoperationshappentobethesamefortwodierentstates,thatdoesnot ithappenstobecorrect,butingeneralthisstepisnotcorrect.intuitively,theproblemisthatjust justifyingthestepwherepreviouslyscheduledstatesare\reused",suchaswhenthepipeliningalgorithm decidedtomaken4thesuccessorofn5.wehavesimplyimpliedthatthisiscorrect,andintheexample byitselfguaranteethatallsubsequentsetsofavailableoperationswouldbethesameinallsuccessorsof Therearethreetechnicallydicultaspectsofthesoftwarepipeliningalgorithm.Therstproblemis therearenoconditionalstatementsorexitsfromtheloop.assumingthatthevariableiisalwayszero thosestates. uponenteringtheloop,notethatstatementsbandcareindependentfortherst50iterationsanddata dependentforthenext50iterations.ifdependenceanalysisrecognizesthatbandcareindependentfor intherst50iterations.followingthepipeliningstrategyforthepreviousexample,repeatingstates therst50iterations,thenastheparallelizedloopisbuilttheschedulercouldplacebandctogether WeillustratethisproblemwiththeloopinFigure4a.Tomaketheexampleassimpleaspossible, wouldbedetectedintheseconditeration,leadingtotheparallelizedprograminfigure4b,whichis 9

10 Section4formalizesthesoftwarepipeliningalgorithmandprovidesconstraintsontheschedulerand clearlyincorrect.inthisexample,irregulardependenciesmakeitdiculttodetectrepeatingbehavior. setsincrementallywasrstpresentedin[en89]forprogramswithoutloops(i.e.withacycliccontrol-ow graphs).insection5,wepresentadetaileddescriptionofthecomputationandmaintenanceofavailable operationsforuseinsoftwarepipeliningofloops.ourpresentationissimplerandeasiertounderstand availableoperationinformationthatguaranteethecorrectnessandterminationofthesoftwarepipelining algorithm. andimplementthanthealgorithmin[en89]. Thesecondproblemiscomputingthesetsofavailableoperations.Analgorithmformaintainingthese tantifthealgorithmistobeusefulinpractice.insection6weshowhowniteresourcesisintegrated directlyonthecorrectnessofoursoftwarepipeliningalgorithm,goodresourceusageisobviouslyimpor- withsoftwarepipelininginoursystem. 4TheSoftwarePipeliningAlgorithm Thethirdsignicantproblemismanagingniteresources.Whileresourceallocationdoesnotbear TheexampleinSection3illustratesthatthekeystepinouralgorithmisdiscoveringwhenstatescanbe \reused"toformasoftwarepipeline.recognizingpatternsinthescheduledoperationsisnottrivialand isinfactnotvalidiftheschedulerandtheavailableoperationanalysisarenotconstrainedinsomeway. Forexample,iftheschedulermerelyselectsoperationstoscheduleatrandom,norepeatingbehaviorcan beinferred.similarly,evenifthescheduleriswell-behaved,theexampleinfigure4showsthatifthe availableoperationanalysisdoesnotexhibitadetectablepattern,softwarepipeliningisnotpossible. Finally,wediscussterminationofthesoftwarepipeliningalgorithm. softwarepipeliningpossible.theseconstraintsarequiteweakandareeasilysatisedinpractice.after presentingtheconstraints,wepresentthesoftwarepipeliningalgorithmitselfandproveitscorrectness. 4.1TheConstraints Inthissectionwepresentconstraintsontheschedulerandavailableoperationanalysisthatmake Denition4.1LetX=f:::;xji usedinthediscussionoftheconstraints. Recallthatxcidenotestheinstanceofoperationxifromiterationcofaloop.Thefollowingdenitionis mustschedulesomeoperationineverystate,and(3)theoperationchosencandependonthesetof operationsavailableandtherelativedistanceiniterationsbetweentheoperationsavailable,butnoton theactualiterationsoftheoperationsavailable.10 specicmachine.thefollowingconstraintrequiresthat:(1)theschedulerisafunction,(2)thescheduler AsdiscussedinSection3,onecomponentofthesoftwarepipeliningalgorithmisaschedulerfora i;:::gbeasetofoperations.thesetxcisthesetf:::;xji+c i;:::g.

11 Constraint4.2LetXbeasetofoperations.Theschedulermustbeafunctionmappingasetof alreadyscheduledoperationsandasetofavailableoperationstoasingleoperationorthevalue\none." Inaddition,schedule(X;A)6=noneifX=;.Wealsorequirethat tion.theoperationxkjreturnedbytheschedulerisanadditionalinstructiontobescheduledinthesame state.theprimaryrestrictionimposedbyconstraint4.2isthattheschedulerisafunctionofoperations availableinthestatebeingscheduled.thisconstraintisweakbecausethesetofavailableoperationsprovidesglobalinformationabouttheprogram theschedulercanchooseanystatementthatcouldbelegally j2ai)_(8ischedule(xi;ai)=none) Inouralgorithm,Xisthesetofoperationsalreadyscheduledinthestatecurrentlyunderconsidera- (9xj8ischedule(Xi;Ai)=xk+i jandxk+i separatestheschedulerfromtherestofthealgorithm,thusisolatingthemostmachine-dependentportion ofthecode.anyschedulersatisfyingconstraint4.2willworkwiththesoftwarepipeliningalgorithm.in Section6weshowhowtogeneralizeConstraint4.2toincluderesourceconstraints. scheduledinthecurrentstate.2thisparticularconstraintalsohasasignicantdesignbenet:itcleanly becausetheoperationsavailableforschedulinginastatencandependonthesetofoperationsalready scheduledinn.forexample,considerthesimpleprogramfragmentinfigure5.assumingthatthe availableoperationsforscheduling.however,theiterativemethoddescribedhereisnecessaryingeneral scheduled.insection3wepresentedasimpliedexampleinwhichtheschedulerchoosesasubsetofthe inginastate.whentheschedulerreturns\none",thestateisnishedandsuccessorsofthestateare Theschedulerisusedbythesoftwarepipeliningalgorithmtorepeatedlyselectoperationsforschedul- whichisincorrect. is,bisnotavailableforschedulinginn0unlessaisscheduledinn0.otherwise,ifthesetofavailable parallelmachineperformsreadsbeforewrites,itisclearthatbothaandbcanbescheduledtogetherin therst(andonly)staten0.however,bcannotbescheduledinn0unlessaisalsoscheduledinn0 that operationsweresimplyfa;bg,thentheschedulercouldchoosetoschedulebinn0andainn0'ssuccessor, iscompletewewishtocomputethesetofavailableoperationsinthesuccessorsofn.theprocedurecall next(n;a)mapsnandatoasetofpairsfhnj;ajigwhereforeverybranchcj2branch(n),njisanew ops(n)[fxigandthenewsetofavailableoperationsgiventhatxihasbeenscheduled.second,whenn Athatareavailableforschedulingassociatedwithastaten.TherearetwowaysthatAcanbeupdated. First,theprocedurecallupdate-one(n;A;xi)returnsapairconsistingoftheupdatedstatewithoperations Asecondconstraintisplacedontheavailableoperations.Atanymomentthereisasetofoperations (empty)node,nj=succ-on-branch(n;cj),andajisthesetofoperationsavailableinnj.implementations ofproceduresupdate-oneandnextaregiveninsection5. thatthisextratheoreticalpowertranslatesintoanypracticaladvantageovertheschemepresentedhere;seesection8. stateatatime,butallstatesatalltimes.becauseschedulingisinherentlyaveryhardproblem,however,itisnotclear 2Onecanimagineevenmorepowerfulschedulers;forexample,aschedulerhavingglobalinformationaboutnotjustone 11

12 Figure5:Asimpleprogram. b:i a:j? i+1 i Constraint4.3ConsideranarbitrarysetofavailableoperationsA,staten,andoperationx.Then thereexistsasetofoperationsbsuchthat 8iupdate-one(m;Ai;xj+i)=hm0;Biiwhereops(m)=ops(n)iand scheduledandtherelativedistance(initerations)betweenoperationsalreadyscheduled,butitcannot Furthermore,thereexistsetsofoperationsAjandstatesnjfor1jjbranch(n)jsuchthat Constraint4.3saysthattheoperationsavailablemaydependonwhichoperationshavealreadybeen 8inext(m;Ai)=fhnj;Aijigwhereops(m)=ops(n)i ops(m0)=ops(n)i[fxj+ig fact,asfarasweknow,everyproposedrepresentationofdependenceinformationsatisesthisconstraint. maintainoperationavailabilityinformation.standarddatadependencegraphssatisfyconstraint4.3,as doextensionstodependencegraphs,suchaslabelingedgeswithconstantdistancevectors[pbj+91].in ofupdate-one,theresultnodeissimplynupdatedtoincludetheoperationxj+i(seesection5.2). dependontheactualvaluesoftheiterationsofoperationsalreadyscheduled.intheimplementation Constraint4.3isneededtoruleoutpathologicalcaseslikeFigure4,whereirregulardependenceanalysis leadstoincorrectschedules. WhetherConstraint4.3issatisedornotdependsontheformofthedatadependenceanalysisusedto Thealgorithmneverbacktrackstoexplorealternativeschedules.Whileabacktrackingversioncouldbe availableoperations(moduloiterationnumbers)asecondtime,itusesthepreviouslyscheduledstate. forallthebranchesofthatstate,andsoon.ifatanypointthealgorithmencountersthesamesetof procedurepipelineinvokestheprocedureschedule-statetobuildasinglestate,andthentobuildstates ThesoftwarepipeliningalgorithmisgiveninFigure6.Givenaninitialsetofavailableoperations,the 4.2TheAlgorithm designedeasily,wefeelabacktrackingalgorithmwouldbetooslowtobepractical. dierenceinthenalparallelprogram.theorderinwhichstatesarescheduledcanmakeadierencein Theorderinwhichpipelineprocessesthesuccessorsofascheduledstateisunspeciedandmakesno 12

13 procedureschedule-state(n;a) procedurepipeline(a) whileschedule(ops(n);a)6=nonedo 8Xscheduled-before[X] returnhn;ai while9hn;ai2tododo letrbeanemptynodeintodo letx=schedule(ops(n);a)in update-one(n;a;x) elselethn;a0i=schedule-state(n;a)and if9js.t.scheduled-before[aj]6=nothen n scheduled-before[aj] todo fhn;aig f:::hni;aii:::g=next(n;a0)in fhr;aig theeciencyoftheavailableoperationscomputation;insection5wepresentaslightlymodiedversion scheduled-before[a] todo Figure6:Thesoftwarepipeliningalgorithm. (todo[f:::hni;aii:::g) fhn;aign ofthealgorithminfigure6thatprocessesstatesinanecientorder. totheoneinfigure6exceptthatitdoesnotreusepreviouslyscheduledstates.letl1bethe(innite) programsemantics.weusetheprograminfigure7toformalizethisintuition.thisprogramisidentical Figure6.LetLbeasequentialloopandletL0betheresultofsoftwarepipelining.WeshowthatLL0. Asarststepintheproof,wemustassumethattheavailableoperationanalysisiscorrect.Intuitively, theavailableoperationanalysisiscorrectifanyschedulethatisconsistentwiththeanalysispreserves WeuseConstraints4.2and4.3toprovethecorrectnessofthesoftwarepipeliningalgorithmin anychoiceofschedulerl1l. parallelprogramdenedbythisalgorithmforaloopl.theavailableoperationanalysisiscorrectiffor Proof:Theproofisbyinductionononthelengthofanexecution.Forthebasecase,lete=hhn0;s0ii Lemma4.4LetL0=pipeline(A)andletL1=pipeline2(A).Forallk,ifthereisanexecution hhn0;s0i;:::;hnk;skiiofl1,thenthereisanexecutionhhn0;s0i;:::;hn0k;skiiofl0. L1isalsoanexecutionofL0. Theessentialstepinprovingthecorrectnessofprocedurepipelineistoshowthateveryexecutionof beanexecutionofl1.considerhowtheinitialstatesofl1andl0arebuilt.theinitialsetofavailable operationsaisthesameforboth.now,inprocedurepipelinewehavescheduled-before[a]=no,because initiallynostatesarescheduled.thenschedule-state(n;a)=hn0;bi.clearlye=hhn0;s0iiisan executionofl0. 13

14 procedurepipeline2(a) while9hn;ai2tododo (Conditionofthewhileisalwaystrue) letrbeanemptynodein lethn;a0i=schedule-state(n;a)and f:::hni;aii:::g=next(n;a0)in fhr;aig hhn0;s0i;:::;hn0i;siiiisanexecutionofl0.furthermore,assumethatthereexistsaksuchthatwhen Fortheinductionstep,assumethate=hhn0;s0i;:::;hni;siiiisanexecutionofL1andthate0= Figure7:Analgorithmthatdenesaninniteparallelprogram. (todo[f:::hni;aii:::g) fhn;aig Akinprocedurepipelinerespectively.Finally,assumethatops(ni)=ops(n0i)k.Itiseasytocheckthat thestatesniandn0iwerescheduled,thesetsofavailableoperationswereainprocedurepipeline2and have alloftheseassumptionsholdafterthebasecase. Thestoresmustbethesameinthetwotransitionssince,byhypothesis,niandn0ihavethesame Ifni=fstopgthenniandn0iarenalstatesandwearedone.Otherwise,inthenexttransitionwe andn0ihavethesameoperationsevaluatedinthesamestore.tonishtheproof,weneedtoshowthat operations.letcbethebranchtakeninstatehni;sii.notecisalsotakeninstatehn0i;sii,becauseni (hni;sii)=hni+1;si+1i algorithm.thatis,wemustshowthatops(ni+1)=ops(n0i+1)jforsomej. ni+1andn0i+1havethesameoperations,possiblydieringiniterationnumbersusedbythepipelining (hn0i;sii)=hn0i+1;si+1i haveschedule-state(m0;bk)=hn0i+1;ckiandops(ni+1)=ops(n0i+1)k. listbypipeline.inl1,letschedule-state(m;b)=hni+1;ci.then,byconstraints4.2and4.3,inl0we cases.fortherstcase,assume8jscheduled-before[bj]=nowhenhm0;bkiisremovedfromthetodo wheremandm0arefresh,emptystatesonbranchcfromniandn0irespectively.nowtherearetwo uled.byconstraint4.3andtheinductionhypothesis,hm;bi2next(ni;a)andhm0;bki2next(n0i;ak) Consideroncemorethestateofthetwosoftwarepipeliningalgorithmswhenni+1andn0i+1aresched- suchthatscheduled-before[bj]=n0.thenn0wasscheduledinl0usingavailableoperationsbj.therest oftheargumentissymmetrictothecaseabove,usingbjinplaceofbkandthefactthatn0i+1=n0.2 sameoperationsavailablefortwostatesimpliesthatallpossiblebranchesfromthosestatesarealsobe Forthesecondcase,assumethat,whenhm0;Bkiisremovedfromthetodolistbypipeline,thereisaj Constraints4.2and4.3areneededtoproveLemma4.4.Theseconstraintsensurethathavingthe 14

15 assumptiontheavailableoperationsanalysisiscorrect.bylemma4.4everyexecutionofl1isalsoan thesame.combiningthecorrectnessconditionfortheavailableoperationanalysisandlemma4.4gives aproofofcorrectness. Theorem4.5IfprocedurepipelineproducesaloopL0fromaninitialloopL,thenLL0. Proof:ToproveLL0wemustshow8s(L;s)=(L0;s).First(L;s)=(L1;s),sinceby pipelineiseventuallyempty.thetodosetdecreasesinsizewhenthereisapairhn;aisuchthatfor showthatitalwaysterminates.toshowtermination,wemustprovethatthetodosetinprocedure 4.3Termination Theorem4.5provesthatthesoftwarepipeliningalgorithmproducesonlycorrectresults,butitdoesnot executionofl0,so(l;s)=(l0;s).2 somej,ajhasbeenscheduledpreviously.letbetheequivalencerelationonsetsofoperations AB,9js:t:A=Bj.Ifweassumethattheprocedureschedule-statealwaysterminates,thento proveterminationitissucienttoshowthatthereareonlynitelymanyequivalenceclassesunder. iftheasetssimplyincreaseinsizeoneachrecursivecall.anecessaryconditionforabisthat isnotnecessarilyterminatingundertheconstraintsgivensofar.consider,forexample,whathappens jaj=jbj;iftherearesetsofunboundedcardinality,thenthereareinnitelymanyequivalenceclasses. Anadditionalconstraintisplacedontheavailabilityinformationtolimitthesizeofthesetofoperations availableforscheduling. Unfortunately,theremaybeinnitelymanyequivalenceclassesandinfacttheprocedurepipeline Constraint4.6ThereisaconstantksuchthatforallpossibleavailabilitysetsA,ifxj2Athenno Lemma4.7Constraint4.6ensuresthatthereareonlynitelymanyequivalenceclassesofsetsofoperationsunder. scheduled,thewindowcannotbeshiftedtoincludeanewiterationattheend. Thisconstraintstatesthatoperationscanbeavailablefromatmostkconsecutiveiterationsatonetime. yh2aforanyhj+k. Proof:IftherearenoperationsinaloopbodyandkconsecutiveiterationscanappearinA,then Thus,theschedulerhasa\slidingwindow"ofoperationsanduntiloperationsintherstiterationare sameforeveryloopscheduled(i.e.,itcanbecomputeddynamically),butitmusthaveamaximumvalue foranyparticularloop.also,itisnotnecessarytomakethewindowanintegralnumberofiterations. Partialiterationsworkjustaswell,althoughthedetailsoftheimplementationareabitmorecomplex. everyavailableoperationsetisasubsetoff:::;xc+ji WhileConstraint4.6ismotivatedbytheneedtoguaranteetermination,italsoleadstoagood ThevaluekofConstraint4.6isaparameterofthesoftwarepipeliningalgorithm.Itneednotbethe i;:::gforsomec,0i<n,and0ji<k.2 implementationoftheprocedurepipeline.themostexpensivepartofpipelineischeckingwhetherthe 15

16 informationforiterationj,andthelastnbitsaresettoreecttheavailabilityofoperationsiniteration currentsetofavailableoperationsahaseverbeenscheduledbeforeforsomeaj.forawindowsizeof j+k.withthisrepresentation,checkingwhetherthesameavailabilityinformationhasbeenseenbefore scheduled(thisoccurswhentherstnbitsareall0)thebitvectorisshiftedleftnbits,discarding 1ifoperationxj+h abitvectoroflengthkn,wherenisthenumberofoperationsinthesequentialloop.thebithn+iis kiterations,operationavailabilityinformationforiterationsjthroughj+k 1canberepresentedas onlyrequirescheckingwhetherthesamebitvectorhasbeenseenbefore,whichcanbeimplementedvery ecientlythroughhashing. iisavailableforscheduling;otherwiseitis0.wheniterationjhasbeencompletely thissectionwegiveanewpresentationofavailableoperations.whilefunctionallyequivalenttothe givenin[en89](forhistoricalreasons,availableoperationsweretermed\uniable-ops"in[en89]).in Availableoperationsanalysisplaysaroleinouralgorithmsimilartotheroleglobaldata-owanalysis playsintraditionaloptimizingcompilers.analgorithmforcomputingavailableoperationswasrst 5AvailableOperations algorithmsof[en89],ourpresentationisbothsimplerandmoredirect,andthenalalgorithmsare easiertoimplement.thedevelopmentisdividedintotwoparts.first,weshowhowtocomputethe initialsetofavailableoperations.second,weshowhowtoincrementallyupdatetheinformationin responsetodecisionsmadebythescheduler.attheendofthesectionweprovethecorrectnessofthe analysisanddiscusssomeeciencyconsiderations. problemofcomputingavailableoperationstoananalysisoflooplessprograms. ofaloop.sinceanynumberofunrollediterationsformaloopless(acyclic)program,werestrictthe Therefore,tocomputetheoperationsavailableforschedulingitissucienttoexamineatmostkiterations RecallthatConstraint4.6forcestheavailableoperationstospannomorethankiterationsofaloop. 5.1ComputingAvailableOperations rithmsinthissectionarepresentedusinganabstractmechanismfordependence.byusingaparticular dependenceanalysisrepresentationthealgorithmscanbemademoreecient.weusethefollowing denitionstomodeldependenceanalysis. anditisbeyondthescopeofthispapertoincludethemhere[kkp+81,fow87,pbj+91].thealgo- manyvariationsondependenceanalysisintheliteraturethatsatisfyourrequirements(constraint4.3) Computingavailableoperationsrequirestheuseofdependenceanalysisbetweenoperations.Thereare Denition5.1Alocationiseitheramemoryaddressoraregister.Foroperationsxandyandsetsof operationsxwedene: 16

17 b:j:=j+1 a:ifa<b f write(x)=thesetoflocationsxmaywrite Figure8:Operationbcankillareferenceliveattheroot. c:h:=j Thesetwrite(x)(resp.read(x))mustincludeeverylocationxcouldeverwrite(resp.read).Theset read(x)=thesetoflocationsxmayread kill(x)=thesetoflocationsxmustwrite write(x)=sx2xwrite(x) becausedependenceanalysismustbeconservativeingeneral;itisnotalwayspossibletoknowatcompiletimeexactlywhichlocationsanoperationmayreadorwrite.thepredicatedepends(x;y)istrueifthere kill(x)mustincludeonlylocationsxalwayswrites.twodierentsetswrite(x)andkill(x)aredened depends(x;y)=write(x)\(read(y)[write(y))6=;depends(x;y)=9x2xs.t.depends(x;y) read(x)=sx2xread(x) kill(x)=sx2xkill(x) maybeadependencefromxtoy. reachablefromstatenthatarenotdatadependentonaninterveningoperation: beforex,resultinginadependenceviolation.thefollowingdataowequationspeciestheoperations ablebecauseofpotentialdatadependenceviolations.assumethatxprecedesyonapathanddepends(x;y) istrue.thenclearlyycannotbeavailableonthatpathuntilxisscheduled,orelseycouldbescheduled Deningcorrectavailableoperationanalysisrequiresidentifyingtheoperationsthatcannotbeavail- SincetheprogramfragmentPbeinganalyzedisloopless,nodeps(n)canbecomputedforallnbyasingle thiscase,operationbcannotbeavailableforschedulingintherststate,becauseitsdenitionofjcould bottom-uptraversalofthecontrol-owgraphforp. TheprograminFigure8illustratesanothersituationinwhichanoperationcannotbeavailable.In nodeps(n)=ops(n)[(([ n02succ(n)nodeps(n0)) fxjdepends(ops(n);x)g) jisliveattherststate,andbcankillc'sreferencetoj.clearly,anyoperationthatcankillalive changethevaluereadbythereferencetojinoperationc.instandardcompilerterminology,location referencecannotbeavailable. readandthereisnointerveningwriteofl.aconventionallivereferenceanalysisisnotsucientforour referencetolocationlisliveatastatenifthereisastatereachablefromnwherelis(potentially) purposes;instead,wewishtocomputelivereferencesdiscountingtheeectofaparticularoperationx. stateoftheprogram.inthiscase,tosay\xhasbeenmovedtotheroot"meansthatalloccurrencesof Moreprecisely,wewishtoknowthesetoflivereferencesassumingthatxhasbeenmovedtotheroot Thesecondcomponentoftheavailableoperationsanalysisisacomputationoflivereferences.A 17

18 xthatcanpotentiallymovetotherootarenotcountedinthelivevariablecomputation.theintuitive justicationbehindthiscomputationisthatwhenmovinganoperationxintheschedule,itisnecessary tocheckifxwillkilllivereferencesinitsnewposition.however,indecidingwhetherornotxwillkill livereferencesinitsnewpositiononeshouldnotcountreferencesofxitselfinitscurrentposition. Thefollowingdataowequationdenesthesetoflocationsliveatstatenmodulooperationx: live(n;x)=read(y)[(([ ThetwocasesinthedenitionofZn0distinguishbetweenthecaseswhereoccurrencesofxcanor wherey=ops(n) fxg wherezn0=(live(n0;x)ifx2nodeps(n0) n02succ(n)zn0) kill(y)) computation(i.e.,live(n0;x)).ifthereisnooccurrenceofxthatcanpotentiallymove,thenalllive bydatadependencies(i.e.,x2nodeps(n0))thenthatoccurrenceofxisdiscountedinthelivereference cannotbeblockedbydatadependencies.ifthereisanoccurrenceofxonapaththatisnotblocked live(n0;stop)otherwise procedurearediscussedattheendofthesection. bottom-uptraversalofthecontrol-owgraphforp.somefurtherimprovementstotheeciencyofthis withthecomputationofnodeps,live(n;x)canbecomputedforallstatesnandoperationsxbyasingle referencesarecounted(i.e.,live(n0;stop)countsallreferences,sincestophasnoeectonthestore).as betheminimumiterationnumberofanyoperationina. threeconditions:xisinnodeps(r),xdoesnotkillanylivereferenceinoperationsotherthanx,andxis inthe\slidingwindow"ofoperations.recallthatconstraint4.6requiresthatoperationsfromnomore thankconsecutiveiterationsbeavailableforanystate.forasetofavailableoperationsa,letmin-it(a) Letrbetheinitialstate(orroot)ofP.Anoperationxisavailableforschedulinginrifitsatises situation.unlikefigure8,thereferencetojinoperationcisnotliveattherootbecauseitreadsthe valuewrittenbyd.theimportantobservationisthatanyoperationthatcankillareferencethatisnot livereferencesataninternalstatenareincludedinnodeps(n).theprograminfigure9illustratesthis Notethatweareconcernedwithlivereferencesonlyattherootr;operationsthatpotentiallykill available(r)=fxijxi2randi min-it(r)<kg liveattheroot(e.g.bcankillc'sreferencetoj)mustbedependentonsomeprecedingoperation(e.g. wherer=nodeps(r) fxjwrite(x)\live(r;x)6=;g writesj;thisoperationpreventsotheroperationsthatcouldwritejfrombeingavailableattheroot. depends(d;b)).thatis,areferencetojthatisnotliveattherootmustbeprecededbyanoperationthat tocomputelive(r;x)foreveryx;itissucienttocomputeitonlyforthoseoperationsinnodeps(r), eciencyofthenaiveproceduredescribedabovecanbeimprovedintwoways.first,itisnotnecessary becausenodeps(r)isasupersetoftheoperationsavailableforscheduling.second,operationsthatkill Themostexpensivepartofcomputingavailable(r)iscomputinglive(r;x)foreveryoperationx.The 18

19 b:j:=j+1 a:ifa<b d:j:=1 f livereferencescouldbedetectedearlierinthecomputationinsteadofcheckingonlyattheroot;wehave notgiventhisalternativetosimplifythepresentation. Figure9:Operationbcankillj,butjisnotliveattheroot. c:h:=j 5.2MaintainingAvailableOperations availableforschedulinginstaterisavailable(r). (seefigure6).thisemptystaterwillbelledwithoperationschosenbythescheduler. Werstdescribeatahighlevelhowavailableoperationsaremaintained,afterwhichwegiveimplementationsoftheproceduresnextandupdate-one.LetPbeasequentiallooplessprogram.Weaddanempty stater(astatewithnooperations)topandmakeittheroot;ristheinitialstateinprocedurepipeline pointintheincrementaldevelopmentoftheparallelizedloop,everyfrontierstateoftheparallelloophas thetodosetandn'ssuccessorsareaddedtothetodoset.astateinthetodosetisafrontierstate.atany statenisscheduled,itisrstlledwithoperations(byschedule-state),andthenhn;aiisremovedfrom ThenextstepistocomputethedataowanalysisofSection5.1.Atthispointthesetofoperations thepropertythatitsknownpredecessorshavebeenscheduledanditssuccessorshaveyettobescheduled. (The\unknown"predecessorsarethosethatareaddedthroughbackedgesinsertedthatcompletethe OncetheinitialglobalanalysisofPiscompleted,wearereadytobeginschedulingstates.When the(modied)programpistheparallelloop. softwarepipelinedloop.)availableoperationsareneededonlyforthefrontierstates;predecessorsof frontierstatesarenevermodied.whenprocedurepipelineterminates,therearenofrontierstatesand loop;theyareusedinsteadtocomputetheavailableoperationsinformationforthefrontierstates. asatree,exceptwherebackedgeshavebeenaddedbypipelining.thestatesbetweenthelinesaandb arethefrontierstates.theseareemptystatesthathaveyettobelledwithoperations.thestatesbelow linebarestatesoftheoriginalsequentialloop.conceptually,thesestatesarenotpartoftheparallel abovethelinelabeledaareparallelstates,alreadyscheduledbythealgorithm.thesestatesarearranged Figure10givesagenericsnapshotofthealgorithm'sdatastructuresduringscheduling.Thestates pipelineselectsapairhr;available(r)ifromtodoandllsitwithoperationsbycallingschedule-state(r;available(r)). Theprocedureschedule-stateinturncallsupdate-oneoneormoretimestochoosethescheduledoper- Thersttodosetisfhr;available(r)ig.Thus,initiallyristheonlyfrontierstate.Theprocedure 19

20 A ations(seefigure6).theprocedurecallupdate-one(available(r);r;x)performstwotasks.first,xis Figure10:Asnapshotofsoftwarepipeliningduringscheduling. B xtorfromitsoriginalplaceinthesequentialschedule.(somecopiesofxmayhavetoremainininterior deletedfrominteriorstatesofpandxisaddedtothefrontierstater.thus,thistransformationmoves statesofpifxcannotmoveonallpathstothefrontierstate;seethediscussionbelow.)whenatestis available(r). andliveareupdatedwherenecessary. movedtorthecontrolowofpmustbemodiedtopreservep'ssemantics.second,thesetsnodeps randthecorrespondingsetsofavailableoperationsai.weimplementnext(r;a)byinsertinganew, ofmaintainingavailableoperationsreasonable.thenewsetofavailableoperationsis(theupdated) restrictedtoarelativelysmallsubsetofthestatesofp;thispropertymakestheincrementalcost Whentheschedulingofriscomplete,next(r;A)=fhri;Aiigisthesetof(empty)successorsriof Animportantfactisthatboththedeletionofxandupdatingofthenodepsandlivesetscanbe amongallelementsofthetodoset.asschedulingproceedstherewillbemultiplefrontierstatesinp, emptystateribeforeeachsucc(r;ci)onbranchci2branch(r).thesetavailable(ri)isexactlytheset P.ThisimplementationofnextallowsP(andthereforetheavailableoperationanalysis)tobeshared ofoperationsavailableforschedulingonbranchcifromr.notethattheriarenewfrontierstatesof Proof:ProcedurenextonlyinsertsemptynodesinP.2 update-one.wecoulddothistriviallyintermsofthelocaltransformationsofpercolationscheduling Lemma5.2LetP0bePwiththemodicationsperformedbynext.ThenP0P. oneforeachelementintodo.animplementationofnextisgiveninfigure11. Tocompletethedescriptionofavailableoperationswemustgiveanimplementationofprocedure 20

21 procedurenext(r;a) returnfhri;available(ri)ijriasdenedaboveg foreachci2branch(r)do letribeafreshemptystate,n=succ-on-branch(r;ci)in nodeps(ri) live(ri;x) succ-on-branch(r;ci) succ-on-branch(ri;hi) Figure11:Implementationofnext. live(n;x)foralloperationsx nodeps(n) n [Nic85],butforcompletenesswedescribeadirectimplementationthatisclosertothewayitshouldbe doneinpractice.letrbeafrontierstateofp,letx=schedule(ops(r);available(r)),andassumethat x6=none.werstdescribehowxisdeletedfrompandaddedtorwhenxisanassignment;thisis theeasiercase.movinganoperationxwhilepreservingp'ssemanticsisalittlesubtlebecausexmay onsomeoperationonthepath;clearly,ccannotbedeletedfromsuchapath.inaddition,theremay otherpathsfromtheroottoc(inthiscasepassingthroughthetruebranchofa)suchthatcisdependent thefalsebranchofa)suchthatcisnotdependentonanyoperationonthepath.however,theremaybe referencesliveattherootandbecausethereisapathfromtheroottoc(inthiscasepassingthrough Figure12aillustratesthissituation.Inthisexample,cisavailableattherootbecausecdoesnotkillany beavailableatthefrontierstatebutstillblockedbydatadependenciesonsomepaths.theprogramin Inthisexample,onlythesinglestatecontainingcneedstobeduplicated,butingeneralmultiplestates rootinfigure12itstillmustbepreservedonpathsfromotherfrontierstates. theoperationbeingmovedissharedbetweenpathswhereitcanmoveandpathswhereitcannotmove. evenbepathsfromotherfrontierstatestoc(representedbytheincomingedgefrome).ifcismovedto ItiseasytoverifythattheprograminFigure12cisequivalenttotheprograminFigure12a. mayhavetobeduplicated.infigure12c,statechasbeendeletedandtheoperationmovedtotheroot. AsillustratedinFigure12b,thisproblemcanberesolvedbyduplicatingstatessothatnoinstanceof innodeps(ni)foreverynionthepath: Astatenkiscoveredbyastaten1foroperationxifthereisapathhn1;:::;nkisuchthatoperationxis tionsandnotation.apathisasequenceofstateshn1;:::;nkisuchthatni+12succ(ni)forall1i<k. Toformalizewhichstatesareduplicatedandwhichstatesaredeletedweneedsomeadditionaldeni- shouldbeleftunchanged.thesimplestcaseisifeverypathtoxiscoveredby(r;x).wesayaloopless afrontierstatetoniscoveredby(r;x).ifpisdeleteconsistentfor(r;x),thenxisnotblockedbydata movedtoafrontierstateritshouldbedeletedonlyfrompathsthatarecoveredby(r;x);otherpaths programpisdeleteconsistentfor(r;x)ifforeveryn2covered(r;x)suchthatn=fxg,everypathfrom Wesayapathiscoveredby(n;x)ifeverystateofthepathisincovered(n;x).Whenanoperationxis covered(n1;x)=fnkjthereexistsisapathhn1;:::;nkis.t.8i;1ikx2nodeps(ni)g 21

22 b:j:=1 a:ifa<0 root:? c:j:=j+1 t? f?d e e@@@rj:=j+1 b:j:=1 a:ifa<0 root:? t? f root:j:=j+1 a:ifa<0 (b)deleteconsistent. d e@@@rj:=j+1 b:j:=1fd t@@@r? dependenciesonanypathtothefrontierstater.hence,wecandeletestatesn=fxgincovered(r;x), Figure12:Movinganassignment. (c)aftercismoved.?d updatethepredecessorsofntopointton'ssuccessor,andaddxtor. Lemma5.3Letxbeanassignmentsuchthatx2available(r)andletN=fnjn=fxgandn2 (Recallthathiistheemptybranch seesection2.) covered(r;x)g.assumethatpisdeleteconsistentfor(r;x).letp0bepwiththefollowingchanges. modifyeachn0wheresucc-on-branch(n0;c)=nforsomen2nsothatsucc-on-branch(n0;c)= r deleteeveryn2n succ(n;hi) Proof:Forbrevityweonlysketchtheproof.Thetransformationcanbeimplementedbyasequence ThenPP0. ofsemantics-preservingpercolationschedulingtransformationsbetweenadjacentnodes[nic85].since eachindividualpercolationschedulingtransformationpreservesprogramsemantics,theentiresequence r[fxg preservesprogramsemantics.2 succ(n0)g.thefollowinglemmagivesaneasytestfordeterminingwhetherpisdeleteconsistent. Lemma5.4Pisdeleteconsistentfor(r;x)in2covered(r;x))pred(n)covered(r;x). looplessprogramdeleteconsistentfor(r;x).thesetofpredecessorsofastatenispred(n)=fn0jn2 Ofcourse,Lemma5.3onlyappliesifPisdeleteconsistent.Wenextshowhowtomakeanarbitrary 22

23 root:? a:a:=1 (a)beforebismoved. c b:ifb<0 t@@@r? fd a:a:=1? root:ifb<0 fa0:a:=1 Proof:Ifeverypredecessorofamemberofcovered(r;x)isincovered(r;x),thenclearlyeverypathfrom Figure13:Movingatest. (b)afterbismoved. c d? theformhr0;:::;n0;ni.thispathisnotcoveredby(r;x).2 andforsomen02pred(n),n062covered(r;x).thentheremustbeapathfromsomefrontierstater0of rton2covered(r;x)iscoveredby(r;x).fortheotherdirection,assumethatthereisann2covered(r;x) followingtwostepsuntilnonischosenin(1): (2)Letn0beaduplicateofnandforeveryp2pred(n)suchthatp62C,ifsucc-on-branch(p;c)=n (1)Choosen2Csuchthatsomepinpred(n)isnotalsoinC. ThefollowingalgorithmmakesPdeleteconsistentfor(r;x).LetC=covered(r;x).Iteratethe consistentretainthenodepsandliveinformationoftheoriginalstate.thesetofstatesforwhichthe Allthatremainsistoupdatethenodepsandlivesets.StatesthatareduplicatedinmakingPdelete analysiscanchangeiscovered(r;x).sincebothnodepsandlivearecomputedbottom-up,theanalysis NotethatthisalgorithmcopiestheminimumnumberofstatesneededtomakePdeleteconsistent. OncePisdeleteconsistentthestepsofLemma5.3canbeappliedtomovextothefrontierstate. thenmodifypsothatsucc-on-branch(p;c)=n0. canbeupdatedinasinglebottom-uppassoverthepathscoveredby(r;x). transformationisillustratedinfigures13.theprograminfigure13aisalreadydeleteconsistentfor update-onetomoveatestxwhilepreservingp'ssemantics,itisnecessarytomodifythecontrolow successoronx'struebranch,theduplicatesetofpathsleadstothesuccessoronx'sfalsebranch.this byupdate-oneisatest.letrbeafrontierstateofp,andletx=schedule(ops(r);available(r)).for ofp.intuitively,weduplicateallthecoveredpathsfromrtox;theoriginalsetofpathsleadstothe Finally,weshowhowtoupdatetheavailableoperationsinthecasewheretheinstructionchosen biisoneoftrueorfalse.thefollowinglemmashowshowtomoveatesttoafrontierstate. preservecontrolow.recallthatabranchcisatruthassignmenttotestshx1=b1;:::;xn=bniwhere (root;b).whenbismovedtorootinfigure13b,stateaisduplicatedonb'strueandfalsebranchesto 23

24 modicationsperformedinorder: Lemma5.5LetPbeaprogramwithfrontierstaterandletx2available(r)wherexisatest.Assume Pisdeleteconsistentfor(r;x)andletX=covered(r;x)inP.ProgramP0isPwiththefollowing (1)Foreachn2X,letn0beastatesuchthatops(n)=ops(n0)andforeachc2branch(n) (2)Foreachn2X,ifsucc-on-branch(n;c)=n1andops(n1)=fxg,thenmodifynsothatsucc-on-branch(n;c)= succ-on-branch(n0;c)=8><>: wheren1=succ-on-branch(n;c). succ-on-branch(n1;hx=truei) succ-on-branch(n1;hx=falsei)ifn12covered(r;x)andops(n1)=fxg n1ifn162covered(r;x) n01ifn12covered(r;x)andops(n1)6=fxg (3)Foreachc2branch(r)wherec=hx1=b1;:::;xn=bniandn=succ-on-branch(r;c)do ThenPP0.3 (4)Letops(r) succ-on-branch(r;hx=false;x1=b1;:::;xn=bni)=(nifn62x succ-on-branch(r;hx=true;x1=b1;:::;xn=bni)=n Proof:Again,forbrevityweonlysketchtheproof.ItiseasytoverifythatP0preservesthecontrol ops(r)[fxg n0ifn2x transformationsbetweenadjacentnodes.2 owofp.asintheproofoflemma5.3,thetransformationcanbeexpressedasasequenceoflocal ofprocedureupdate-oneisgiveninfigure14. modiestheoriginalstatesincovered(r;x)sothattheyleadtothetruebranchofx.part(3)modiesthe branchesofrtopointtotheoriginalnodesifxistrueandtothecopiednodesifxisfalse.adescription andbyassigningsuccessorssothatthepathsformedbythen0leadtothefalsebranchofx.part(2) Theimplementationwehavedescribedissomewhatnaiveandthereareinecienciesthatcanbe Part(1)ofLemma5.5duplicatescoveredpathsbycreatingacopyn0ofeverystateincovered(r;x) eliminatedatthecostofgreatercomplexityinthealgorithm.mostofthepotentialproblemsarerelated code. tospaceexplosion,eitherinthesizeofthenalcodeorinthesizeofintermediatedatastructuresused Whenperformedonthestatesoftheparallelschedule,thisoptimizationreducesthesizeofthenal bythealgorithm.somestatesthatareinitiallydierentmaybecomeidenticalasaresultofscheduling operations.thisobservationappliestobothstatesintheparallelscheduleandstatesthathaveyetto bescheduled.agoodimplementationshouldmergestatesthatareidenticalandareonidenticalpaths. becausecontrol-owisredirectedaroundthem. 3NotethatinthisconstructiontheoriginalstatescontainingxarenotremovedfromP,buttheybecomeunreachable 24