LoadBalancingforMinimizingExecutionTimeofaTargetJobon anetworkofheterogeneousworkstations DepartmentofElectricalandComputerEngineering S.-Y.LeeandC.-H.Cho sylee@eng.auburn.edu Auburn,AL36849. AuburnUniversity NOWs.Insuchanenvironment,partitioning(loadbalancing)atargetjobbasedononlythe executiontimeofatargetjobistobeminimized.jobarrivalrateandsizeare\random"ona Anetworkofworkstations(NOWs)maybeemployedforhighperformancecomputingwhere Abstract tominimizeexecutiontimeofatargetjobonasetofworkstationswheretheround-robinjob schedulingpolicyisadopted.ithasbeenveriedthroughcomputersimulationthattheproposed rstordermoments(means)ofsystemparametersisnotoptimal.inthispaper,itisproposed toconsiderthesecondordermoments(standarddeviations)alsoinloadbalancinginorder staticanddynamicloadbalancingschemescansignicantlyreduceexecutiontimeofatarget jobinanowsenvironment,comparedtocaseswhereonlythemeansoftheparametersare used. scheduling,standarddeviation,staticloadbalancing,stochasticmodel KeyWords:Dynamicloadbalancing,Executiontime,Networkofworkstation,Round-robinjob

tributedcomputingtoolinanincreasingnumberofcases[1][2].accordingly,usinganows Anetworkofworkstations(NOWs),orcomputers,isbeingemployedasahighperformancedis- 1 Introduction issueofloadbalancingingeneralisnotnew.manyresearchershaveinvestigatedvariousaspects balancinghasasignicanteectonperformanceonecanachieveonanowsenvironment.the ecientlyforspeedingupvariousapplicationshasbecomeanimportantissue.inparticular,load ofloadbalancingforquitelongatime[3][4][5][6][7][8][9][10]. (orheterogeneity)ofsystem,loadbalancingoverhead,speciccharacteristicsofajob,etc.in processorspeed,jobarrivalrateandsize,communicationamongjobs(orsubtasks),homogeneity Thereweremanyparametersandcharacteristicsconsideredinloadbalancing.Theyinclude mostofthepreviouswork[11][12],onlythemeansofsuchparameterswereusedinloadbalancing. However,aparametermayhavethesamemeanforallworkstations,butquitedierentavariance onadierentworkstationinaheterogeneousenvironment.also,inmanycases[13][14][15],the emphasiswasonbalancingjobdistributionratherthanminimizingexecutiontimeofatargetjob. onanindividualworkstation.thisismainlyduetothefactthateachworkstationisusuallyshared forms,especiallydedicatedtightly-coupledmultiprocessorsystems,i.e.,randomnessofjobarrival ThereisafeatureofNOWs,whichdistinguishesitfromotherhighperformancecomputingplat- workstationtime-dependent.thatis,thenumberofjobsonaworkstationistobemodelledasa Thisrandomnessinjobarrivalandsizemakesthenumberofjobssharing(theprocessoron)a bymultipleindependentuserswhosubmittheirjobsatanytime.also,thesizeofajobisrandom. ofworkcompletedineachjobduringatimeintervaldependson(e.g.,inverselyproportionalto) randomvariable.whentheprocessorissharedamongjobsinaround-robinfashion,theamount workstationdoesnotachievetheminimumpossibleexecutiontime.aswillbeshownlater,not distributing(loadbalancing)atargetjobconsideringonlythemeanofthenumberofjobsoneach thenumberofjobs(sharingtheprocessor)inthatinterval.onanetworkofsuchworkstations, takeintoaccountthestandarddeviationofthenumberofjobsoneachworkstationinadditionto onlythemeanbutalsothestandarddeviationofthenumberofjobsaectsexecutiontimeofajob itsmeaninloadbalancing. onaworkstation.therefore,inordertominimizeexecutiontimeofatargetjob,itisnecessaryto analyticallyanddemonstratedviasimulationthatthesecondordermomentsaswellastherst ordermomentsofparametersaretobeusedtominimizeexecutiontimeofatargetjobona Inthispaper,asarststeptowarddevelopinganecientloadbalancingscheme,itisshown networkofheterogeneousworkstations.workstationsareconsideredtobeheterogeneouswhenthe partitionedforloadbalancingandcommunicationisnotrequiredamongsubtasks. time-sharedbymultiplejobs.inthisearlystudy,itisassumedthatatargetjobcanbearbitrarily meanandstandarddeviationofthenumberofjobsvarywithworkstation.eachworkstationis Themaincontributionsofthisworkare(i)derivationofanalyticformulasofperformance 1

measuresusedinloadbalancing,(ii)designofstaticanddynamicloadbalancingschemesfor thestandarddeviationsaswellasthemeansofparameterscanoutperformthoseconsideringthe meansonly. minimizingexecutiontimeofatargetjob,and(iii)showingthataloadbalancingschemeutilizing derivedanalyticallyonasingleworkstation,whicharetobeusedforloadbalancingonmultiple workstations.insection4,theproposedstaticanddynamicloadbalancingstrategiesaredescribed. InSection2,astochasticmodelofworkstationsisdescribed.InSection3,asetofmeasuresis strategies.insection6,aconclusionisprovidedwithremarksonthefuturedirections. InSection5,resultsfromextensivecomputersimulationarediscussedtovalidatetheproposed Glossary 2 AStochasticModelofWorkstations Thefollowingnotationsareadoptedinthispaper. Wiworkstationi X thenumberofworkstations Xi thesizeofatargetjobtobedistributed Ai theportionofxassignedtowi thenumberofjobs(randomvariable)arrivedinanintervalonwi ni aithestandarddeviationofai themeanofai Ni themeanofni arrivinginthecurrentinterval thenumberofjobs(randomvariable)inanintervalonwi,excludingthose nithestandarddeviationofni Si thesizeofjob(randomvariable)arrivingatwi i sithestandarddeviationofsi theservicerate(computingpower)ofwi themeanofsi Ti ti executiontime(randomvariable)measuredinintervalsonwi themeanofti Or Oc overheadinvolvedinredistributingload overheadinvolvedincheckingloaddistribution thestandarddeviationofti Whenavariableistobedistinguishedforeachtimeinterval,asuperscriptwithparentheses E[Z]expectationofZ willbeused,e.g.,a(j) idenotesaiforthejthinterval.thesubscriptofarandomvariable,which 2

isusedtodistinguishworkstations,isomittedwhenthereisnoneedfordistinction,e.g.,asingle workstationorwhenitdoesnotvarywithworkstation. ServicePolicy Atimeintervalisaunitoftimeforschedulingjobs(sharingtheprocessor)onaworkstation.All timemeasuresareexpressedinintervals.itisassumedthateachworkstationadoptsaround-robin foreachjobinanintervalonworkstationi(denotedbywi)whereiistheservicerateofwi. schedulingpolicy.aworkstation(processor)spends,oneachjobinaninterval,anamountoftime Thosejobsarrivedinanintervalstarttobeserviced(processed)inthefollowingintervalwithout whichisinverselyproportionaltothenumber(ni)ofjobsinthatinterval.thatis,i niisallocated JobArrival anydistinctiondependingontheirarrivaltimes(aslongastheyarriveinthesameinterval).itis tobenotedthatniincludesalljobsarrivedbutnotcompletedbyii 1. Jobsaresubmittedatrandomtimeinstancesand,therefore,thenumberofjobsarrivinginan intervalmaybemodelledbyarandomvariabledenotedbyai.themeanandstandarddeviation JobSize ofaiaredenotedbyaiandai,respectively. standarddeviation(si).itisassumedthatthejobsizeisindependentofthejobarrivalrate. Overheads Thesizeofajobvarieswithjobandmayhaveacertaindistributionwithamean(Si)anda suchastheremainingportionofxionwi,andalsoaconstantoverhead,or,forredistributingthe Loadsonworkstationsarecheckedtodetermineifloadbalancingistobedone.Itisassumedthat, givenanumberofworkstations,thereisaxedamountofoverhead,oc,forcheckinginformation remainingxoverworkstationswhenitisdecidedtoperformloadbalancing. beestimatedfromagivennetworkofworkstations.also,nicanbedirectlymonitoredinpractice. thesystemparametersmaybeknowninsomecases.ortheirmeansandstandarddeviationscan Thejobarrivalrateandjobsizewillbereferredtoassystemparameters.Thedistributionsof distributionofeachofthesystemparameters. Itneedstobenotedthattheproposedloadbalancingschemesdonotassumeanyparticular loadbalancingschemesformultipleworkstations,arederived. 3Inthissection,certain(performance)measuresonasingleworkstation,tobeusedintheproposed PerformanceMeasuresonaWorkstation 3

characteristics(morespecically,themeansandstandarddeviationsofthesystemparameters)do notvarywithtimeonaworkstation,itissaidthattheworkstationisin\steadystate".when Thejobofwhichexecutiontimeistobeminimizedisreferredtoastargetjob.Whentheload theyvarywithtime,theworkstationissaidtobein\dynamicstate". Thenumberofjobs,n(j),mayberelatedtothejobarrivalrate,a(j),asfollows. 3.1NumberofJobs n(j)=1+a(j 1)+(n(j 1) 1)p(j 1) intervalandtherstterm(of1)correspondstothetargetjob. wherep(j 1)istheprobabilitythatajobinthe(j 1)thintervaliscarriedovertothejth (1) dependsonandthedistributionsofa(j)ands(j),fromequation1, NotingthatE[n(j)]=E[n(j 1)]=N,andlettingPdenotethesteadystatevalueofp(j)which Also,thestandarddeviationofn(j)canbederivedasfollows. N=1+ 1 P A (2) 3.2ExecutionTime n=qe[(n(j) N)2]= 1 P a (3) Inthejthinterval,thetargetjob(anyjob)isprocessedbytheamountof tintervalstocompletethetargetjob,txi=1 n(j)forallj.ifittakes Then,1 Let'sexpressn(j)asN+n(j).Then,E[n(j)]=0sinceE[n(j)]=N,andE[(n(j))2]=2n. n(i)=x: (4) asfollows. n(j)canbeapproximatedbyignoringthehigherordertermsbeyondthesecondorderterm BytakingE[](expectation)onbothsidesofEquation4withEquation5incorporatedinto, n(j)= 1 N+n(j)1N0@1 n(j) 1 N + n(j) N!21A (5) themeanofexecutiontimeofthetargetjob,t,canbederived. 4

NotethatTdependsonnotonlyNbutalsonbothofwhichinturndependonthestandard T= 1+2n NXN2 (6) deviationsaswellasthemeansofthesystemparameters,a,a,s,ands(andofcourse).note thatexecutiontimeofatargetjobonaworkstationwitharound-robinjobschedulingdecreases approximationusedtoobtaint,thestandarddeviation,x,ofxcanbeshowntoben thatisprocessed(completed)inthejthinterval.thenx(j)= asvariation(n)inthenumberofjobsincreases. Inordertoderivethestandarddeviationofexecutiontime,letX(j)denoteaportionofX N+n(j).Followingthesimilar amountoftargetjobprocessedovertintervals(whichisthemeanexecutiontimeofthetarget Now,assuming\uncorrelatedness"ofX(j)betweenintervals,thestandarddeviation,X,ofthe N2. job)canbeeasilyshowntobeptx.finally,thestandarddeviationofexecutiontimeofa targetjobmaybederived(approximated)bydividingxbythemeanprocessingspeedwhichis XTandusingEquation6.Thatis, t=xt X=pTq1+2n n NN2 (7) 4.1StaticLoadBalancing 4 LoadBalancingoverHeterogeneousWorkstations Intheproposedstaticloadbalancingscheme,atargetjobispartitionedsuchthatthefractionof XassignedtoWifori=1;;WisinverselyproportionaltotheexpectedexecutiontimeofX onwiwherewisthenumberofworkstationsavailableforx.lettidenoteexecutiontimeofthe targetjobonwi(i.e.,whenwionlyisemployedfortheentirex).then, Ti= i1+2ni NiX LetXidenotethesizeoftheportionofXtobeassignedtoWi.Then,Xiisdeterminedas N2i fori=1;;w: (8) follows. Xi= PWi=11Ti XTi isconstantforalli.thisloadbalancingstrategyassignsmoreworktoaworkstationwitha Notethat,evenwhenNiisthesameforalli,Xwouldnotbedistributedevenlyunlessni (9) 5

largervariationinthenumberofjobsonitwhentheaveragenumberofjobsisthesameforall rateof1+3=23onw2.therefore,alargerportionofthetargetistobeassignedtow2which and3).then,atargetjobwouldbeprocessedattheaveragerateof2onw1whileattheaverage workstations.supposethatn1=n2=2,n1=0,andn2>0(say,n2alternatesbetween1 hasalargervariationinthenumberofjobs. 4.2DynamicLoadBalancing point). ofatargetjob)andhowfrequentlyloaddistributionistobechecked(determinationofchecking Twoessentialissuesindynamicloadbalancingarehowloadshouldberedistributed(redistribution Redistribution LetXi:remdenotethesizeoftheremainingportionofatargetjobonWibeforeloadbalancingat 6.Thatis, completiontimeofthetargetjobwouldbemaxifti:remgwhereti:remiscomputedusingequation acheckingpoint.ifloadbalancing(redistribution)isnotdoneatthecheckingpoint,theexpected Ti:rem= i1+2ni NiXi:rem N2i fori=1;;w (10) Xrem=PIXi:rem,isredistributedoverWiaccordingtoEquation9.Thatis,X0i:rem= Now,ifloadbalancingistobedoneatthecheckingpoint,thetotalremainingjob,ofwhichsizeis PWi=11 Ti:rem Xrem foralli;jsinceloadhasbeenbalanced. wherex0i:remisthesizeoftargetjob(portion)tobeassignedtowiafterloadbalancing.the expectedexecutiontimeofx0i:remonwiisdenotedbyt0 i:rem.then,notethatt0 i:rem=t0j:rem Ti:rem Theexpectedreduction,T,inexecutiontimeofthetargetjobcanbeexpressedas whereoristheoverheadforredistribution. T=maxifTi:remg T01:rem Or (11) DeterminationofCheckingPoint executiontime,t)exceedsacertainthreshold,i.e.,tthreshold. Loadbalancing(redistribution)iscarriedoutonlywhentheexpectedbenet(reductionin whichanywiisidle(doesnotworkonthetargetjob)beforethetargetjobiscompletedonall Inordertominimizeexecutiontimeofatargetjob,itisnecessarytominimizethedurationin 6

stillworkonthetargetjob. Wi.Hence,theloaddistributionistobecheckedbeforeanyofWibecomesidlewhiletheothers Equation7.Areasonableestimateofthe\highlylikely"earliestcompletiontimeonWimaybe approximatedtobet0 Thestandarddeviation,t0i:rem,ofexecutiontimetocompleteX0i:remcanbederivedusing setasfollows. Then,thenextcheckingpoint,Tcheck,measuredwithrespecttothecurrentcheckingpointis i:rem ht0i:remwherehisatuningfactorcloseto1. Tcheck=minifT0 tionbeforeanyofwicompletesitsexecutionoftargetjob(x0i:rem).notethatonceawicompletes Thatis,intheproposeddynamicloadbalancingscheme,itisattemptedtocheckloaddistribu- i:rem ht0i:remg (12) 5X0i:remitwillnotbeutilizedforthetargetjobatleastuntilnextcheckingpoint. Anextensivecomputersimulationhasbeencarriedoutinordertoverifythereductioninexecution timeofatargetjob,whichcanbeachievedbyconsideringthesecondordermoments(standard SimulationResultsandDiscussion geneousworkstations. 5.1Simulation deviations)aswellasthemeansofsystemparametersforloadbalancingonanetworkofhetero- similartrendshavebeenobservedforallthreedistributions,onlytheresultsforthe(truncated) distributions,havebeenconsideredforeachofthejobinter-arrivaltimeandthejobsize.since Inthissimulation,threedierentdistributions,i.e.,exponential,uniform,andtruncatedGaussian eachwithadierentseed,andthenresults(executiontimeofatargetjob)wereaveraged. testedforawiderangeofeachparameter.theprogramwasrunmultipletimesineachtestcase, Gaussiandistributionareprovidedinthispaper.Theproposedloadbalancingschemeshavebeen \SE"(StaticEven)whichdistributesatargetjobevenlyinthestaticloadbalancingand\SM" \staticproposed"and\dynamicproposed",respectively)arecomparedtootherapproaches,i.e., Theproposedstaticanddynamicloadbalancingschemes(\SP"and\DP"whichreads (StaticMean)and\DM"(DynamicMean)whichuseonlythemeans(Ni)ofthenumberofjobs inthestaticanddynamicloadbalancing,respectively. totheexecutiontimeforeachcheckingpoint.ifloadisredistributed,or(redistributionoverhead) isalsoadded(foreachredistribution). Inthecasesofdynamicloadbalancing,theoverhead,Oc,forcheckingloaddistributionisadded 7

Executiontimeismeasuredinintervals.Inadditiontoexecutiontimeofatargetjob,themeasure of\relativeimprovement"isusedincomparison,whichisdenedasris=tsm TSP 5.2ResultsandDiscussion staticloadbalancingwheretsmandtspareexecutiontimesofatargetjobachievedbysmand SP,respectively.Similarly,therelativeimprovementisdenedforthedynamicloadbalancingas RID=TDM TDP anddp,respectively. Resultsforcaseswhereworkstationsareinthesteadystatearediscussedrst. wheretdmandtdpareexecutiontimesofatargetjobachievedbydm discussedinsection3.2,tclearlyshowsitsdependencyonnanddecreasesasnincreases. nfordierentninasteadystate.obviously,tincreasesasnincreases.moreimportantly,as InFigure1,executiontimeofatargetjobonasingleworkstationisplottedasafunctionof expected,theproposedloadbalancingschemes(spanddp)workbetterthantheotherschemes, comparedamongtheveloadbalancingschemesmentionedabove.first,itcanbeseenthat,as InFigure2,(parallel)executiontimeofatargetjobinsteadystatesontwoworkstationsis conrmingthatthesecondordermomentsofthesystemparametersaretobeconsideredinload balancinginordertominimizetheexecutiontimeofatargetjob.second,itneedstobenotedthat SPachievesshorterexecutiontimesthanDP.Thisisduetothefactthatinasteadystatetheload characteristicsdonotvaryinthelongtermandthereforetheone-timeinitialloadbalancing(by SP)isgoodenough,andthatDPpaystheoverheadofcheckingandbalancingduringexecution. inn,leadingtoashorterexecutiontime(refertoequation6).however,increasingshasan executiontime(figure2-(b)).thisisbecauseanincreaseinacausesalargerincreaseinnthan Third,anincreaseinaleadstoashorterexecutiontime(Figure2-(a))whilethatinstoalonger oppositeeect. analyzedinasteadystate.asshowninthegure,whentheoverheadsarerelativelylow,dpcan stillachieveashorterexecutiontimethanthatbysp.however,astheoverheadsbecomelarger, InFigure3,dependencyofexecutiontimeontheloadcheckingandredistributionoverheadsis thansp. theystarttoosetthegainbythedynamicloadbalancingandeventuallymakedpperformworse DMinFigure4-(b).Itcanbeseeninbothcasesthattherelativeimprovementincreasesasthe dierenceinaorsbetweentwoworkstationsbecomeslarger.thisisduetothefactthatthe TherelativeimprovementbySPoverSMisconsideredinFigure4-(a),andthatbyDPover agiandssiareaandsoftheithgroupofworkstations.itcanbeobservedthattherelative largerthedierenceis,thelessaccuratetheloadbalancingbysmbecomes. improvementbyspoversmincreaseswiththenumberofworkstations.itincreasesmorerapidly Eectsofthenumberofworkstations,W,areanalyzedforsteadystatesinFigure5where theloadbalancingbyspbecomesmore(relatively)accuratethanthatbysmasthedierence whenthedierenceineitheraorsislarger.again,theseobservationsstemfromthefactthat inthesecondordermomentsbetweengroupsofworkstationsgrows. 8

45 40 N=2 (A=1, S=20) N=3 (A=2, S=10) N=4 (A=3, S=5) 35 Execution time 30 Figure1:ExecutiontimeofXononeworkstationwhere=100andX=1000. 25 20 15 10 0 10 20 30 40 50 60 σ (%) n 34 32 S_E S_M S_P D_M D_P 36 34 32 S_E S_M S_P D_M D_P 30 30 28 28 26 26 S2=20,andX=2000.(a)a1=53%,s1=30%,s2=48%,(b)a1=53%,a2=45%,s1=30% Figure2:Parallelexecutiontimeontwoworkstationswhere=100,A1=1,A2=2,S1=10, (b) 24 24 22 22 Now,acasewhereasystemparametervarieswithtimeisconsidered,i.e.dynamicstate.In 0 20 40 60 80 0 20 40 60 80 σ a2 (%) σ (%) thestaticschemes(spandsm).also,itisnotedthatdpwhichtakesa1intoaccountachieves Figure6,a1varieswithtimesuchthatitsdeviationfromthevalueusedbySPischanged(larger forcaseiwithlargeri).asexpected,thedynamicschemes(dpanddm)performbetterthan s2 ashorterexecutiontimecomparedtodm.theimprovementbydpoverdmtendstoincrease 6withthedeviation. astherstordermoments(means)ofsystemparametersbetakenintoaccountforloadbalancing Inthispaper,ithasbeenproposedthatthesecondordermoments(standarddeviations)aswell ConclusionandFutureStudy onatime-sharedheterogeneousparallel/distributedcomputingenvironment.theseloadbalancing schemeswhichattempttominimizeexecutiontimeofatargetjobhavebeentestedviacomputer simulation.ithasbeenveriedthatconsideringthesecondordermomentsalsoinbothstaticand 9 Execution time Execution time

46 44 42 S_E S_M S_P D_M D_P Execution time 40 38 S2=30,X=2000,a1=90%,a2=0%,s1=96%,s2=96%. Figure3:Parallelexecutiontimeontwoworkstationswhere=100,A1=1,A2=2,S1=20, 36 34 32 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 O c, O r 12 10 σ s2 =1 % σ s2 =47 % σ s2 =95 % 13 12 11 σ s2 =1 % σ s2 =46 % σ s2 =96 % Relative improvement (%) 10 8 9 6 8 7 4 6 A1=0:5,A2=0:5,S1=40,S2=40,X=3000,a1=0%,s1=100%,(b)Relativeimprovement, 5 2 RID,byDPoverDMontwoworkstations.=100,A1=2,A2=2,S1=20,S2=20,X=16000, Figure4:(a)Relativeimprovement,RIS,bySPoverSMontwoworkstations. 4 a1=0%,s1=96%,oc=0:1,or=0:1. 0 3 0 20 80 0 20 80 σ (%) σ a2 a_2 (%) (reductioninexecutiontimeofatargetjob)becomeslargerasthedierenceinthesecondorder NOWswitharound-robinjobschedulingpolicyadoptedineachworkstation.Theimprovement dynamicloadbalancingcanleadtoasignicantreductioninexecutiontimeofatargetjobona momentsbetweenworkstationsorgroupsofworkstationsincreases.thesimilarobservationshave application. schemesaresimpleandgeneral,andthereforearebelievedtohaveagoodpotentialforwide beenmadeforallofthethreedistributionsconsideredforeachsystemparameter.theproposed performanceanalysisforrealworkloadsonanows,etc. Thefuturestudyincludesconsiderationofcommunicationamongsubtasksandjobgranularity, 10 Relative improvement (%)

Relative improvement (%) 25 20 σ a_g2 =50 %, σ s_g2 =0 % σ a_g2 =50 %, σ s_g2 =96 % σ a_g2 =76 %, σ s_g2 =0 % σ a_g2 =76 %, σ s_g2 =96 % 15 Ag1=0:5,Ag2=0:5,Sg1=40,Sg2=40,X=1500,ag1=50%,sg1=96%. Figure5:Relativeimprovement,RIS,bySPoverSMonmultipleworkstationswhere=100, 5 0 2 4 6 8 12 W 50 45 S_E S_M S_P D_M D_P Execution time 40 Figure6:Parallelexecutiontimeontwoworkstationswhere=100,A1=1,A2=2,S1=20,S2=30, 35 25 References X=2000,a1=24%,a2=0%,s1=0%,s2=20%.a1varieswithtimewhereitsdeviationfrom thevalueusedbyspislargerforcaseiwithlargeri. 1 2 3 4 [2]Pf,\InSearchofClusters". [1]D.Culler,\ParallelComputerArchitecture",MorganKaufman,1999. [4]C.PolychronopoulosandD.Kuck,\GuidedSelf-SchedulingSchemeforParallelSupercomputers",IEEETransactionsonComputers,vol.36,no.12,pp1,425-1,439,December1987. [3]G.Cybenko,\DynamicLoadBalancingforDistributedMemoryMultiprocessors",J.Parallel anddistributedcomputing,vol.7,pp279-301,1989. [5]S.Ranka,Y.Won,andS.Sahni,\ProgrammingaHypercubeMulticomputer",IEEESoftware, pp69-77,september1988. 11

[7]M.Cierniak,W.Li,andM.J.Zaki,\LoopSchedulingforHeterogeneity",Proc.ofthe4th [6]M.MaheswaranandH.Siegel,\ADynamicMatchingandSchedulingAlgorithmforHeterogeneousComputingSystems",Proc.HeterogeneousComputing'98,pp57-69,1998. [8]A.GerasoulisandT.Yang,\OntheGranularityandClusteringofDirectedAcyclictask IEEEInternationalSymposiumHigh-PerformanceDistributedComputing,pp78-85,August graphs",ieeetransactionsonparallelanddistributedsystems,vol.4,no.6,pp686-701, 1995. [9]S.M.FigueiraandF.Berman,\ModelingtheSlowdownofData-ParallelApplicationsin 1993. [10]J.C.JacobandS.-Y.Lee,"TaskSpreadingandShrinkingonMultiprocessorSystemsand 1998. HomogeneousClustersofWorkstations",Proc.HeterogeneousComputerWorkshop,pp90-101, [11]E.P.MakatosandT.J.Leblanc,\UsingProcessorAnityinLoopSchedulingonShared- NetworksofWorkstations",IEEETransactionsonParallelandDistributedSystems,vol.10, no.10,pp1082-1101,october1999. [12]S.SubramaniamandD.L.Eager,\AnitySchedulingofUnbalancedWorkloads",Proc.Supercomputing'94,pp214-226,1994. no.4,pp379-400,april1993. MemoryMultiprocessors",IEEETransactionsonParallelandDistributedSystems,vol.5, [13]M.-Y.Wu,\OnRuntimeParallelSchedulingforProcessorLoadBalancing",IEEETransactionsonParallelandDistributedSystems,vol.8,no.2,pp173-186,February1997. [14]X.ZhangandY.Yan,\ModelingandCharacterizingParallelComputingPerformanceOnHeterogeneousNetworksofWorkstations",Proc.ofthe7thIEEESymp.ParallelandDistributed [15]B.-R.TsaiandK.G.Shin,\Communication-OrientedAssignmentofTaskModulesinHypercubeMulticomputers",Proc.ofthe12thInternationalConferenceonDistributedComputing Processing,pp25-34,October1995. Systems,pp38-45,1992. 12