Application-LayerAnycasting

Transcription

1 Application-LayerAnycasting SamratBhattacharjee MostafaH.Ammar GIT{CC{96/25 EllenW.Zegura ZongmingFei VirenShah nicatewiththe\best"server,accordingtosomeperformanceorpolicycriteria,inagroup ofcontent-equivalentservers.weexaminethedenitionandsupportoftheanycasting signedtosupportserverreplicationbyallowingapplicationstoeasilyselectandcommu- ageographicallywide-spreadnetwork.theanycastingcommunicationparadigmisde- Serverreplicationisakeyapproachformaintaininguser-perceivedqualityofservicewithin Abstract Weexaminetheuseofvarioustechniquesformaintainingthisinformationatanycast basedlters(e.g.,serverresponsetime)andpolicy-basedlters;wefurtherallowltering processismetricinformationdescribingtherelativeperformanceofreplicatedservers. bothattheanycastresolverandlocaltotheanycastclient.akeyinputtotheltering fromnetwork-layersupport,ourdenitionincludesthenotionoflters,functionsthatare appliedtogroupsofaddressestoaecttheselectionprocess.weconsiderbothmetric- intooneormoreipaddressesusinganycastresolvers.inadditiontobeingindependent paradigmattheapplicationlayer,providingaservicethatmapsanycastdomainnames resolvers. GeorgiaInstituteofTechnology Atlanta,Georgia30332{0280 CollegeofComputing

2 theprovisionofsuchservicesistheabilityofaservicetomeetthedemandsofalargenumber 1Introduction ofusersthataregeographicallywide-spread.itisalsoimportantthattheuser-perceived Asthisviewbecomesmoreprevalent,itbecomesimportanttoprovide,withintheInternet, explicitsupportfortheecientdeliveryofnetworkedservices.animportantconsiderationin TheInternetisincreasinglybeingviewedasprovidingservices,andnotjustconnectivity. qualityofservice(e.g.,responsetime,throughput,reliability)bemaintainedatanacceptable scalabilityoftheservice. [4]andmulticastingofserverresponsesoverthenetwork[5,6].Inthispaperweconsider andcompetitive(inthecaseofcommercialservices)level.thisisoftenreferredtoasthe theserverreplicationtechnique.inparticular,weinvestigatetheanycastingcommunication paradigmwhichhasbeenproposed[7]tosupportserverreplication. service.theseincludeserverreplication[1],caching[2,3],batchingofrequestsattheserver Asoriginallydened[7],anycastingprovides: Therehavebeenseveralapproachesproposedforimprovingthescalabilityofanetworked Inthisdenition,anIPanycastaddressisusedtodeneagroupofserversthatprovidethe sameservice.asenderdesiringtocommunicatewithonlyoneoftheserverssendsdatagrams withtheipanycastaddressinthedestinationaddresseld.thedatagramisthenrouted tosomecriteria)identiedbytheanycastaddress. usinganycast-awarerouterstoatleastoneoftheservers(presumablythe\best"according \astatelessbesteortdeliveryofananycastdatagramtoatleastonehost,and Inourworkweadoptamoregeneralviewofanycastingasacommunicationparadigm preferablyonlyonehost,whichservestheanycastaddress." providingtheanycastingservicedenitionandexaminingtheprovisionofthisservicewithin theanycastingservice1.theoriginalanycastingproposal[7]can,therefore,beviewedas wedierentiatebetweentheanycastingservicedenitionandtheprotocollayerproviding hasthefollowinglimitations: thatisanalogoustothebroadcastandmulticastcommunicationparadigms.inparticular, theiplayer. plicationlayer.ourmotivationderivesfromthefactthatnetwork-layer-supportedanycasting SomepartoftheIPaddressspacemustbeallocatedtoanycastaddresses.ForIPv4, Inthispaperweexaminethedenitionandsupportoftheanycastingparadigmattheap- classofaddresses.ipv6proposals[9]doincludeaspecicaddressspaceallocatedto natingsomeexistingaddressesasanycast(e.g.,withinclassc)orcreatingaseparate severalwaystoallocateanycastaddresseshavebeensuggested[7].theseincludedesig- 1Forexamplemulticastingasacommunicationparadigmrepresentsadesiretosendthesamemessagetoagroup Theselectionoftheservertowhichananycastaddresseddatagramissentismade Theuseofanycastaddressesrequiresroutersupport.Routersmustrecognizeanycast addressesandforwardthemproperly.routersmustcoordinatewithoneanotherto entirelywithinthenetworkwithnooptionforuserselectionorinput. ensurethatdeliveryisusuallymadetoexactlyonehost. anycasting. ofreceivers.themulticastparadigmcanbesupportedusingmulticastroutingatthenetworklayeroritcanbe supportedaboveaunicast-onlynetworkortransportlayerbyusingmultipleunicasts.indeedthislatterapproach washowmanymulticastproblemswereaddressedinearlysystemdesigns(e.g.,theoriginalisisdistributedsystem [8]).Thereisclearlyadistinctionbetweenmulticastasacommunicationparadigmandhowthisparadigmis supported. 1

3 FIGURE1AnycastNameResolutionQuery/ResponseCycle Application Client Anycast Domain Name Specification Client Filter IP Address Anycast Server Filter Anycast Group Response Metric Info Query Anycast Anycast Resolver Filter ConsistentwiththestatelessnatureofIP,thedestinationisdeterminedonaperdatagrambasis.Thus,twosuccessivedatagramssenttoananycastaddressmaybe deliveredtotwodierenthoststhatservetheaddress.aprotocolthatrequiresalldatagramstobedeliveredtoasinglehostcanuseanycastingfordiscoveryofasatisfactory host,butsubsequenttransmissionsshouldusestandardiptotheselectedhost. Becauseofthenetworklayer'sabilitytoecientlydetermineshortestpaths,itiswellsuitedforananycastingservicebasedontheselectionoftheclosest(innumberofhops orotherdistanceordelaymetrics)server.anapplicationlayerapproachisbettersuited athandlingavarietyofothermetricssuchasserverthroughputorturnaroundtime. Whereasnetwork-layersupporthingesaroundtheuseofanycastIPaddresses,ourapplicationlayersupportmakesuseofanycastdomainnames(ADNs).ThefunctionofanapplicationlayeranycastingserviceisthustomapanADNintooneormore(multicastorunicast)IP addresses.animportantfeatureofapplication-layeranycastingisthatitdoesnotrequire modicationstonetworklayeroperations. Thispaperfocusesonthedesignofaninfrastructuretoprovideanapplication-layeranycastingservice.OurdesigncentersaroundtheuseofanycastresolverstoperformtheADN toipaddressmapping.clientsinteractwiththeanycastresolversaccordingtoabasic query/responsecycleillustratedinfigure1:aclientgeneratesananycastquery,theresolver processesthequeryandreplieswithananycastresponse.akeyfeatureofthesystemisthe presenceofmetricdatabases,associatedwitheachanycastresolver,containingperformance dataaboutservers.theperformancedatacanbeusedintheselectionofaserverfroma group,basedonuser-speciedperformancecriteria.weconsidermetricdeterminationtechniquesthatcanbeusedtomaintaintheanycastresolverdatabases.inaddition,weinvestigate howanycastclientapplicationsmayinterfacewiththeanycastresolver. Therestofthepaperisstructuredasfollows.Section2discussesrelatedwork.Sec- 2

4 denitionsofreplicatedservice,anycastgroupsanddomainnames,andanycastresolverfunctionality.insection4weelaborateontheanycastquery/responsecycle,discussingthedesign tion3describesinmoredetailthecomponentsofapplication-layeranycasting,providingour 2RelatedWork thenetworkortheresolvers.weconcludethepaperinsection6withanassessmentofthe meritsofapplication-layeranycastingandadiscussionoffuturework. andoperationoftheinterfacetoananycastresolver.section5considersasetofmetricsand metricdeterminationmechanismstoallowaccuratemeasurementswhilenotundulyloading adecade.initially,withlowtomoderateserverloads,theproblemwashowtondthe Theserverorresourcendingproblemhasbeenthesubjectofmuchinvestigationforover desiredresourceoverthenetworkknowingonlyitsnameorproperty.manytechniqueswere proposedandinvestigated.theseinclude:1)theuseofmulticastorbroadcastcommunication ServiceLocationProtocolwhichallowsausertospecifyasetofserviceattributeswhichcan recently,theservicelocationworkinggroupoftheietfisconsideringthedesignofthe atsiteswheretheresourceisfrequentlyaccessed[15].thisearlyworkdealtwithsituations wherethereistypicallyasingleinstanceoftheresource.thecaseofamobileresourcewas addressedthroughinterestingtechniquessuchastheuseofforwardingaddresses[16].more theresource(e.g.,[12,13,14])and3)theuseofcachingofaresource'slocation(notcontent) to\touch"allthelocationswheretheresourcemayresideinanattempttondit(e.g., beboundtoaserver'snetworkaddressinadynamicfashion[17]. [10,11]),2)theuseofvariousnameserverarchitecturesinordertolookupthelocationof MendezandMilliken[7]proposingtheideaofanycastinganddiscussingitsnetwork-layer supportand2)arecentstudybyguytonandschwartz[18]whichaddressestheproblem equivalentservers.twonotablestudiesinthisareaare:1)theoriginalworkbypartridge, ndingproblem:itisnowimportanttondthe\best"serverfromamongmanycontent- withtheworld-wideweb,theinternethasexperiencedadramaticgrowthintheuseand resultinginthedesiretoreplicate(ormirror)servers.thisaddsanewdimensiontotheserver provisionofinformationservices.thishasresultedinheavydemandsbeingplacedonservers Beginningwithinitialserviceslikeftp,archie,andgopherandculminatingmorerecently oflocatingthenearestserver.thelatterworkalsopresentsaclassicationof\best"-server locationschemes.theworkisrelatedtoearlierworkontheharvestsystem[19]which usedintheharvestsysteminordertoimprovethescalabilityoftheservice.anotherrelated theguytonandschwartzclassicationwhichconsiderstheoriginaldenitionofanycastingas andsearchingthroughthisinformation.toolsforcachingandreplicationofindicesarealso providesasetoftoolsforgatheringinformationfromvariousserversandecientlyindexing projectisthesonarnetworkproximityservice[20]inwhichtheauthorsdeneaservice whichcanreturntheclosest(inhops)serverfromamongaprovidedlist. anetwork-layer-supportedservice.ourworkinvestigatesthecompletedesignofapplicationlayeranycastingandnotjustthemetricprobingaspectasdiscussedin[18].wealsoconsider Bychoosingtodeneanycastingasacommunicationparadigm,wedeviatesomewhatfrom Theconceptofprobingthenetworkandtheserverstodetermineperformancemeasuresis aclosermatchtotheapplicationrequirement. usingavarietyofmetrics(notjusthop-distanceasprovidedbysonar)inordertoprovide 3

5 relatedtovarioustools,andsystemsthatareusedfornetworkmanagementpurposes.examplesoftheseincludethetraceroute[21],andmwatch[22]toolsandmanagementsystemsthat servicedubbedontimedelivery2whichprobesawebsiteatregularintervalsandprovides integratesuchtoolssuchasthefremontsystem[23].remotemeasurementandmonitoring areportontheperformanceofthewebsiteasperceivedbytheirclient. ofsystemperformancehasalsobeenexploredaspartoftheextensiveworkondistributed systemmonitoring[24,25].recently,thecompanytimedancersystemshasbeenoeringa observations.thisisrelatedtothepushcachingidea[26]whereserversareinchargeof theideaofhavingtheserver\push"ontothenetworkitsownlocallygeneratedperformance pushingthedesiredinformationontoremotecachesandtheserverpushmechanism[27] implementedinnetscapebrowsers. server.modicationstothednsresolutionmechanismsaremadetoallocatetheloadamong suchaschememultipleserversareclusteredandappeartotheoutsideworldasinglelogical Toavoidthescalabilityproblemsinherentinprobingforperformancewealsoexplore theservers. 3GeneralizedApplication-LayerAnycasting Finally,anycastingisrelatedtothetechniqueusedtobuildscalableHTTPservers[28].In ofthisparadigm. 3.1WhatIsaReplicatedService communicationparadigm.latersectionsareconcernedwithissuesrelatingtotherealization Inthissectionwedescribeinmoredetailthedenitionofapplication-layeranycastingasa Ananycastdomainname(ADN)isusedtoidentifyaparticularnetworkservicethatcanbe canbeconstruedasbeingreplicatedservers.dependingontheapplication,thewebservers thoughinourdenitionreplicatedserversdonotnecessarilyhavetobeidentical. iftheycontainequivalentcontentand/orfunctionalityfromanapplicationperspective.itis truethatserverswithidenticalcontentorfunctionalityareconsideredreplicasofeachother, providedbymultiplelocationsonthenetwork.wesaythattwoserversreplicateeachother forcnninteractive,timemagazineandusatodaycouldconstituteasetofreplicated primaryexampleisquorumconsensusapplicationsthatariseindistributedsystems.quorum serversaswell. websearchsites(e.g.,lycos,infoseek,yahoo)carrydierentinformation,they,nevertheless, cated,whereallserversprovideidenticalfunctionality.ontheotherhand,whereasthemany Insomecasesmultipleserversarerequiredtoprovideasingleinstanceofaservice.A TheNetworkTimeProtocol(NTP)isanexampleofasimpleservicethatiswidelyrepliplications,suchasthereadingofstrongly-consistentreplicateddata,mayrequirepermission coordinatornodes.ifmutuallyexclusiveexecutionofoperationsisdesired,e.g.,aswould operationneedstocollectpermissionfromamajorityofthecoordinatornodes.otherap- berequiredwhenupdatingstrongly-consistentreplicateddata,thenthenodeexecutingthe Anoperationmayproceedtocompletiononlyifitisgrantedpermissionfromanumberof consensusisageneralclassofsynchronizationprotocolsfordistributedsystems[29,30,31,32]. 2Seehttp://

6 fromacertainnumberofnodes,notnecessarilyamajority.byincludingallthecoordinator nodesinamulticastgroupwithasinglegroupaddress,multicastcommunicationcanbeused torunaparticularcomputation.thesetofavailablehostscanthenbemadetoforman content,e.g.,computeservers.inthiscasetheuserisinterestedinndingaserveronwhich reasons:faulttoleranceandloadbalancing. byaquorumcollectortocommunicatewiththesetofcoordinatornodes. theappropriatequorumtobeabletodelivertheservice.thisreplicationisdonefortheusual nodeslargerthantherequiredquorum.thisallowsanysubsetofthecoordinatornodeswith anycastgroup.theperformancemeasureofinteresttoselectthebestserverwouldnormally Wealsoextendournotionofservicetoincludemoregeneralfunctionalitythaninformation Thistypeofserviceistypically\replicated"bymakingthethenumberofcoordinator becpuload.aninterestingvariationiswheretheuserhasaparallelprogramtorunthat requirestheuseofmultiplehosts.thisisanotherinstanceofmultiplehostsprovidingasingle instanceofaservice;makingtheanycastgroupacollectionofpre-formedmulticastgroups eachcontainingenoughhoststoexecutetheparallelprogram. thegrouptothe\best"addressinthecollectionaccordingtosomecriteria. groups4.inbothcases,theanycastingserviceprovidesamappingfromtheadnrepresenting grouprepresentsagroupofserversandinthelatteritrepresentsacollectionofmulticast 3.2AnycastGroups Ananycastdomainname(ADN)uniquelyidentiesa(potentiallydynamic)collectionofIP unicastipaddressesorentirelyofmulticastipaddresses3.intheformercasetheanycast addresses,whichconstitutesananycastgroup.ananycastgroupcanbemadeupentirelyof metric. theclassofquorumconsensusapplicationsthatwedescribedabove.fortheseapplications,it addressesinthegrouprepresentstheaddressofoneofthereplicatedserversprovidingthe concerned),andtheanycastingservicemakesadecisionbasedonperformanceand/orpolicy service.reachinganyoneofthemisacceptable(asfarasservicecontentandfunctionalityis forwardandderivesfromtheoriginalmotivationoftheanycastingparadigm.eachoftheip Weallowacollectionofmulticastaddressestoformananycastgroupinordertosupport ThemotivationforcollectingasetofunicastIPaddressesintoananycastgroupisstraight- makessensetodenemulticastgroupsmadeupofsubsetsofthesetofallcoordinators,assign eachsubsetamulticastaddressandcollectthemulticastaddressesintoananycastgroup.the applicationthenwouldusetheadnassociatedwiththisgrouptorefertothecoordination service.theanycastingimplementationwouldthenmaptheadnintoamulticastgroup withthedesiredquality(e.g.,leastaveragecoordinatorloadordistancefromclient). thispapersincewecannotconceiveofanapplicationthatwoulduseit. aliases)5.inthiscasetheanycastingserviceprovidesamappingfromanadnintoahost facilitatetheprocessofselectingfromamongthegroupmembersaswillbediscussedlater. lookup.ontheotherhand,knowingthedomainnamesthatmakeupananycastgroupmay domainnameoralias.obtainingtheipaddressinthiscaserequiresanadditionaldnsserver 3Althoughthegeneralcaseofamixtureofunicastandmulticastaddressesispossible,wedonotconsideritin 4AfurthergeneralizationofourdenitionallowsthecollectionofanycastIPaddressesintoananycastgroup Itisalsopossibletodeneananycastgroupasacollectionofserverdomainnames(or representedbyanadn.werelegateconsiderationofthis\nestedanycasting"tofuturework. anycastgroupisacollectionofhostsandnotacollectionofgroupsofhosts. 5Becausethereisnoexistingconventionfornamingmulticastgroupsthiscurrentlyonlymakessenseifthe 5

7 FIGURE2AnycastResolverArchitecture Local Anycast Resolver 3: Request for ADN X Members and Metrics Authoritative Anycast Resolver for ADN X 5: Caches ADN X 2: Determines Authoritative X = IP addr. 1 Membership and Metrics; IP addr. 2 Initiates Metric Collection... IP addr. 3 systemingeneral,andtheanycastresolverarchitectureinparticular.weproposeanapproach 3.3AnycastDomainNamesandResolverArchitecture 4: List ADN X Members and Metrics 1: Request ADN X 6: Response DNS[12],allowingustoconsiderissuesrelatedtoanycastresolverdesignseparatelyfrom Internetinfrastructure. inthispaperthatisderivedfromtheinternetnaminganddirectoryservicearchitecture.such anapproachmakesitstraightforwardtointegrateouranycastingarchitectureintotheexisting Theissueofthestructureofanycastdomainnamesinuencestheoperationoftheanycasting withtheoperationofdns. othernameserviceissues.inreality,thefunctionsofananycastresolvercouldbeintegrated Weprefertoviewtheanycastresolveraslogicallydistinctfromothernameserverslike Ananycastdomainname(ADN)isoftheform<Service>%<DomainName>.Sucha Client guredwiththeaddressofitslocalanycastresolver(inthesamewaylocaldnsserversare oftheadnidentiestheservicewithintheauthoritativeresolver. namewilltypicallybeusedasanargumenttoalibrarycallthatinvokestheanycastingservice indicatesthelocationoftheauthoritativeanycastresolverforthisadn.theservicepart andresultsinthemappingofthisadntoanipaddress.thedomainnamepartofthesystem localresolver.determiningtheaddressoftheauthoritativeanycastresolverforaparticular partoftheadnandobtainstheanycastgroup'sinformationwhichisthencachedinthe ADN,itcanprocessthequeryimmediatelyandreturntheappropriaterepsonse.Otherwise, thelocalresolverdeterminestheaddressoftheauthoritativeresolverforthedomainname congured).ananycastclientmakesitsinitialanycastquerytoitslocalresolver.ifthe resolverisauthoritativefortheadninthequeryorifithascachedinformationaboutthe TheanycastresolverarchitectureisshowninFigure2.EachnetworklocationisprecondomaincanbedoneusingthesametechniqueusedforDNStodetermineanauthoritative nameserver[12]. Thishierarchicalnamingallowsuserstodenetheirownanycastgroupsandmaintain 6

8 suchgroupsinlocalanycastresolvers.bypropagatingtheadnofalocally-denedanycast group(includingthenameofthedomaininwhichitsauthoritativeresolverresides)allows otherstomakeuseofthisanycastgroup. ADNtoIPaddress.Thisinformationincludes: 1.ThelistofIPaddressesthatformparticularanycastgroups.Authoritativeresolvers 2.Themetricinformationassociatedwitheachmemberoftheanycastgroup.ThisinformationismaintainedindependentlyateachanycastresolverthathastheADNgroumation,metricsmaintainedattheauthoritativeresolverare,ingeneral,oflittlevalue membershipinformationcached.becauseofthelocalsignicanceofthemetricinfor- Ananycastresolvermaintainstheinformationnecessarytoperformthemappingfrom maintainthedenitivelist,whereaslocalresolverscachethisinformation. 1.Theanycastgroupmembershipinformationisupdatedaccordingtosomemembership Theinformationmaintainedintheresolverisupdatedusingtwoseparatemechanisms: astheybegintogathertheirownmetricinformation. memberlistforagivenadn.localresolverscanusethisinformationas\hints"initially informationwheneveritreceivesarequestfromanotherresolverfortheanycastgroup tootherresolvers.theauthoritativeresolvermayprovideitslocallymaintainedmetric shouldbe)dependentontheparticularsoftheservice.forexample,thelistofftp protocol.wedonotdiscussthedetailsofsuchaprotocolherebecauseitcan(and locationsforapopularwebbrowsercanbedisseminatedfromahomelocationtovarious 4InteractingwithAnycastResolvers 2.Theserviceperformancemetricinformationwouldtypicallyneedtobemaintaineddynamically.WeproposeanddiscusssomemechanismstoachievethisinSection5. beadministrativelyconguredinanycastservers. anycastservers.forthewebsearchservice,ontheotherhand,thelistofserverscan First,theanycastclientgeneratesaquerythatispassedontotheanycastresolver.After processingthequery,theresolvergeneratesaresponsewhichissentbacktotheclient.in general,anadditionalprocessingstepisperformedattheclienttoyieldthenalipaddress Ananycastresolverisconsultedthroughtheuseofanycastqueriesandtheresolverresponds withanycastresponses.thebasicanycastingquery/responsecycleisillustratedinfigure1. areappliedtotheinformationmaintainedabouttheanycastgrouptoobtainanipaddress. result. Wedistinguishbetweenthreebasictypesoflters:content-independentlters,metric-based Inourproposedapproachtheanycastingserviceisaccomplishedthroughasetofltersthat 4.1FilteringandDecisionMaking resolver.aseparate(butstraightforward)protocolneedstobedenedforsuchanexchange. Anothermessageexchangemayberequiredbetweenalocalresolverandanauthoritative 7

9 ltersandpolicy-basedlters.wefurtherdistinguishbetweentwolocationsforltering: withintheresolverandattheuser.(seefigure1.) oftheanycastgroupofsomegivensize. basedsolelyonmembershipinformationandnotbasedonanyothercriteriamaintainedor knownbytheserver.examplesofsuchltersinclude:1)theselectionofanymemberat random,2)theselectionofallmembersoftheanycastgroup,or3)theselectionofasubset ciatedwiththemembersoftheanycastgroup.thefollowingvariationsareofinterest: Content-independentlterscanbeusedtospecifyaselectionofanycastgroupmembers Selectthebestanycastgroupmember(s)accordingtoafunctionofoneormoremetrics. Selectthebestanycastgroupmember(s)accordingtoasinglecriterion. Metric-basedltersspecifyselectionaccordingtothevaluesofoneormoremetricsasso- Selectthebestanycastgroupmember(s)resultingfromthesequentialapplicationof Forexample,aweightedsumofmetricscanbeused,allowingaclienttocontrolthe importanceofeachmetricintheoverallevaluationofananycastgroupmember. thebroadrangeofothercriteriathatmightaecttheselectionofanaddress.policylters Policy-basedltersarenotbasedonmeasurementsofperformance,butratherencompass metricisusedtofurtherrenetheselectionset.forexample,onemightrstselecta weblocationthathasthefastestresponsetime,andthen(amongstthefastest)choose thelocationthatistheleastnumberofhopsaway. process.inthecaseoftwolters,therstmetricisgiventoppriorityandthesecond lters.bycomposingltersinseries,onecanspecifyastrictpriorityintheselection arelikelytobeboolean,inthesensethatanaddresseithermeetsorfailsthepolicycriteria. Policylterstendtoruleoutgroupsofaddresses;forexample,onemightbeinterestedin thenapplypolicy,however,someothercompositionofthetwotypesofltersmayalsobe useful.applyingpolicyltersmaybeeasierifthemembershipoftheanycastgroupisknown accessingallserverscontainingincometaxinformationthatarenotrunbytheirs.perhaps bydomainnamesratherthanjustipaddress.iftheanycastgroupmembersaremaintained themostnaturalinteractionofmetricandpolicy-basedltersistorstapplymetricsand cantaketheresolverlistandfurtherapplylterstoselectasinglemember. byipaddress,mappingtheseaddressesintodomainnames(e.g.,throughtheuseofdns beginswiththesetofanycastgroupmembersandapplieslterstoproducealist.theclient aretwolocationsforlters:intheanycastresolverandattheanycastclient.theresolver providetheanycastclientwithalistofaddressesthatmeetthespeciedcriteria.thusthere PointerQueries)mayberequiredbeforetheapplicationofapolicylter. wheretheyareapplied.inourgeneralizedversionofanycasting,theanycastservicemay privacyregardingtheaddressselectionprocess.followingthetaxexampleabove,onemay Theprovisionofltersattheclientallowsformorelocalcontrol,includingincreased Relatedtotheissueoftheorderinwhichltersareappliedistheissueofthelocation notwanttomakeknowntothesystemthefactthatirs-runserversaretobeavoided.thus thepolicylterthatexemptsirs-runserverscouldbeappliedlocallyratherthanspecied tothesystem.anotherexamplecouldbetheuseofalocally-preferredsubsetofserversin thenusedasanalltertotheresultsobtainedfromtheanycastingservice. theformofalocally-denedanycastgroup.themembershipofthislocally-denedgroupis 8

10 theltersexercisedinthesystemwillbepredominantlymetric-based. membersissuccessfulifthenumberofaddressesproducedisacceptabletotheclientprocedure.inallcasesthereturnofexactlyoneipaddressforanapplicationtouseshouldbe Obviously,thenecessaryinformationmustexistatthelocationwherethelterisapplied. SuccessfulFilteringWesaythatasequenceofltersappliedtothesetofanycastgroup Itseemslikelythattheltersexercisedattheuserwillbepredominantlypolicy-based,while anacceptableoutcome.ideally,theapplicationprocedurethatinvokestheanycastingservice Thisisbecause,ingeneral,itisnotalwayspossibletoknowaprioritheoutcomeofthe shouldalsobecapableofdealingwiththecasewheremultipleornoaddressesarereturned. applicationofasetoflterstoananycastgroup.inthecasewheremultipleaddressesare Incasetheanycastserviceinvocationreturnsnoaddresses,afallbackpositionneedstobe returnedfromtheanycastserviceinvocation,theapplicationcanarbitrarilypickoneofthe programmedintotheapplicationprocedure.forexample,itcouldretrywithanothersetof returnedaddresses.thisisequivalenttoapplyingyetanother(content-independent)lter. 4.2FilterSpecication Wenowturnourattentiontotheissueofhowaclientmayspecifyalterfortheanycast ltersoracontent-independentlteraskingforanygroupmemberatrandom. resolvertouse.localltersarerelativelyeasytodealwithsincetheyaretypicallyconceived, speciedandappliedinthesamelocation.resolverlters,ontheotherhand,areconceived availablethiswaythroughwell-knownidentiers.inthesecondtypeoflterspecication, theformofafunctionofmetricsoraproceduredescribingsomeelaboratesequenceoflter alterthatisalreadybuiltintotheresolver.inthiscasealltheclienthastodoisrefer theclientprovidesaproceduraldescriptionoftheoperationofthelter.thiscanbein tothislterusingsomeagreeduponidentier.weexpectthatmanypopularlterswillbe applications.aninterestingpossibilityistheuseofjava[33]astheprogramminglanguageto ataclientbutarerunbytheresolver,whichaddscomplexitytotheirspecication. specifylters,withthecommunicationofthejavaprogramtotheanycastresolverincluded Weenvisiontwotypesoflterspecications.Intherstcase,theclientdesirestouse Argumentstothiscallcanbeusedtospecifythelteringdesired,eitherbyidentifyinga thatcanbeusedbyanapplicationtoinvoketheprocessofadntoipaddressmapping. alternatives.intherst,anewapplicationlayerinterface(api)functioncallisprovided, aspartofanycastclient/resolverinteraction. mannersimilartohowthegethostbyname()functioncalltypicallyinvokesthednsservice. server.anexampleofthisinthecontextofthesocketsapiisthedenitionofa(new) getanyhostbyname()functioncallthatcanbeusedtoinvoketheanycastingserviceina built-inlterorbypointingtosomelocalfunctionorproceduretobecommunicatedtothe Anotherimportantissueiswhereandhowlterspecicationsaremade.Weexploretwo Names(MQ-ADN)toconveythedesiredlteraspartoftheADNbeingsenttotheanycast server.ametric-qualiedadnisoftheform<filter-specification>.<adn>where Filter-Specificationprovidesinformationabouttheltertobeusedbytheserver.This isrelativelystraightforwardifabuilt-inlterisbeingspecied.forexample,thename AnalternativemethodforspecifyingltersistouseMetric-QualiedAnycastDomain 9

11 insuchcasestohavethemq-adnsgeneratedautomaticallyusinglibraryfunctionsthatare madeavailableattheapi. complexurlsgeneratedbycgiprograms[34]toperformdatabasesearches.itisnecessary ADN.Usingmetric-qualiedADNsismorecomplexwhenaproceduraldescriptionofthelter isdesired.onepossibilityistoallowcomplexfilter-specicationinthemq-adn,akintothe ServerLoadbuilt-inlterontheanycastgrouprepresentedbythewwwsearch%cc.gatech.edu ServerLoad.wwwsearch%cc.gatech.educouldconveytotheresolverthedesiretousethe needtopickanftpserverandthenissuethecommand: >ftpftp.freebsd.org applicationsneednotberewritten.manyoftheseapplicationstakedomainnamesasinput; oftheanycastingservice.inthiscase,themq-adnisaugmentedbythesux\.any". onewouldsimplyneedtosubstituteamq-adninstead,toallowtheapplicationtomakeuse UsingtheaugmentedMQ-ADN,itwouldbepossiblefortheusertoissuethecommand: Forexample,auserwantingthemostrecentversionofFreeBSDusingftpwouldordinarily MQ-ADNscanbeenhancedtoallowexistingAPIcallstobeleftintact,thusexisting software,withthemembershipoftheanycastgroupmaintainedatthecc.gatech.edudomain's anycastresolver. whereftpfreebsdisananycastgroupconsistingofserversthatcontainthedesiredfreebsd >ftpthroughput.ftpfreebsd%cc.gatech.edu.any case,thenananycastresolverqueryisformulated,afterparsingthenametoobtainthe callandcheckiftheargumentisanadn(i.e.,endswiththe.anysux).ifthatisthe 4.3AnImplementationusingAugmentedMetric-QualiedADNs softwareinthisinstance. Inourimplementation(showninFigure3)weinterceptcallstothegethostbyname()socket WenextexplainhowourprototypeimplementationallowstheuseofunmodiedFTP theusualprocedureiscalleddirectly.thisallowsustousetraditionalapplicationswithout resolverlterspecication.thisqueryissenttotheanycastresolverwhichreturnsasetof 5MetricsandMetricDetermination modicationandgivestheoptionofusingthemwithorwithouttheanycastingfeature6. returnthedesiredipaddress.ifthegethostbyname()argumentisotherthananadnthen hostdomainnames.thesetispassedthroughaclientltertoreduceittoasinglehost domainname.thisnameisthenusedasinputtotheoriginalgethostbyname()procedureto metricdatawillnottypicallybesevere;ratherthanselectingthe\best"server,theservice values.further,theperformancepenaltyassociatedwithout-of-dateorslightlyinaccurate mayidentifya\nearly-best"server.asapracticalmatter,evenmetricsthataresuciently formancemetricswith\reasonable"accuracy,withoutundulyloadingthenetworkorthe servers.notethatthemeasurementsneednotbeperfectlyaccurate;theanycastmechanismreliesuponrelativeorderingtodeterminethebestserver,ratherthanabsolutemetric Theuseofmetric-basedltersintheanycastingservicerequirestheabilitytomeasureper- library.wesuccessfullyusedthistechniquetoaugmentmanyapplicationswithanycastingcapabilities,including weareunabletosubstituteourownlibrarysubroutines. themosaicwebbrowser.however,someapplications,suchasthenetscapebrowser,arestaticallylinked,andthus 6Useofthistechniqueonexistingapplicationsrequiresdynamiclinkingoftheapplicationcodetoourown 10

12 FIGURE3TheStructureofanImplementationusingMetric-QualiedADNs Gethostbyname other.any accuratetoallowonetoavoidtheworstoftheservers,e.g.,thosethataredownorcurrently Anycast Server accessingreplicatedservers.wegiveexamplesofmeasurementstakenonvarioussystems; thesemeasurementsindicatethatsome,butnotall,metricsandserversexhibitsucient Client Filter performancevariationtowarrantanycasting.wenextturntotheissueofcollectingmetric unreachable,willmakeforausefulservice. Webeginthissectionbyexaminingasetofmetricsthatmaybeofinteresttoclientsin Domain Name theloadthattheyplaceonthenetworkandservers,andintheaccuracyofthemeasurements. DNS Weexaminethreeofthemethods proxyprobing,userexperienceandserverpush in information.wedescribefourpossiblewaystomaintaininformation;thesemethodsdierin moredetailwithinanexampleenvironment.weconcludebysummarizingtheperformance andsystemoverheadcharacteristicsofeachofthemethods. IP Address serverresponsetime,2)server-to-userthroughput,3)serverload,and4)processorload7.we focusonthesemetricsbecausetheyarelikelytobeofinteresttoclientsinselectingfromaset batch,real-time)andthepreferencesoftheusers. measured.metricsofinterestwilldependonthecharacteristicsoftheservice(e.g.,interactive, 5.1Metrics ofreplicatedservers.further,theyrepresentdiversityinthecomponents(server,path,client) involvedindeterminingthemetricvalueanddiversityinthetimescaleonwhichthemetric Thearchitecturedescribedthusfarissucientlygeneraltosupportanymetricthatcanbe valuechanges.weenvisionthatadditionalmetrics(e.g.,packetloss,delayjitter)wouldalso 7Theoriginalanycastingproposal[7]wasprimarilyconcernedwithaserverdistance(hopcount)metric;measuringserverdistancehasalsobeenstudiedextensivelybyGuytonandSchwartz[18]. Tobetterunderstandhowanycastingwillperform,weexaminefourspecicmetrics:1) 11

13 FIGURE4HTTPResponseTimeatNews-OrientedServers 4.5e+07 5e+07 ESPNet CNN FN USA Today CNN 4e e+07 beofinteresttocertaintypesofapplications. 3e+07 aresignicantdierencesacrossserversatvarioustimes,otherwisetheselectionofaserver 2.5e+07 couldjustaswellberandom.further,thevariationinametricmustoccuronatimescale 2e+07 thatispracticaltotrackusingmeasurementtools.wenowpresentexamplemeasurements foreachofthefourmetrics,withtheaimofbetterunderstandingwhichmetricsareamenable 1.5e+07 toanycasting. 1e ServerResponseTime Theuseofmetricstoidentifythe\best"serverispredicatedontheassumptionthatthere 5e+06 browsing.wemeasureserverresponsetimebymeasuringtheround-triptimeforaquery8. Serverresponsetimeisofinterestforinteractiveclient-serverapplicationssuchasweb 7:15 A.M. 2:00 P.M. 20:50 P.M. WehavemeasuredHTTPserverresponsetimeatfournews-orientedservers Time EST9.Thex-axisintheplotistheprobeindex;they-axisisthemeasuredresponsetimein workstationsonourownresearchgroup'ssubnet.atypicalresultisdepictedinfigure4, espnet.sportszone.com, forvariousperiodsoverseveralweeks.themeasurementsweretakenfromrelativelyunloaded representingservermeasurementsevery5minutesduringa13hourperiodfrom8:00to21:00 constructed;mostoftenweused\anycasttest(testversion)"followedbyoneormorecarriagereturns.aprotocol specicconnectionwasinitiatedtotheserver'sspecicserviceport(tcpport80incaseofhttpservers).the microseconds.each25ticksonthex-axisisapproximatelytwohours. orlunchtimeoverthethreeustimezones.thecnnnewsserverisalsoactiveinthelater afternoon.theseresultsdemonstrateseveralordersofmagnitudedierencebetweenthebest timetocreatetheconnection,sendthequery,andthetimefortheservertoprocessandrespondtothequerywere andworstserversatagiventime,andalsodemonstratethatnooneserverisconsistently 8Specically,aquerystringconsistingofasetofcharactersnotlikelytobeintheserver'svocabularywas Theseserversshowtime-of-daytrends,withsignicantactivityfrom11:30to15:00EST, measuredusingtheunixgettimeofday()systemcall.theserverresponsewasparsedtomakesuretheserver tripnetworklatencyandthetimefortheservertoforkaprocessandrespondtothequery.inthecasewhere actuallyrespondedtothequery,andtimeouts(ifany)werelogged.thus,thequeryservestomeasuretheround theserverloadislight,thepathlatencywilldominatethismetric.notethatasimilarmethodcouldbeusedto 12 Response Time (Microseconds)

14 FIGURE5FTPThroughputunderVaryingLoad 35 ATM Ethernet timemetric. best.theseserversappeartobefairlygoodcandidatesforanycastingbasedonaresponse fromourmeasurementmachinetotheserverssharemanylinks,thuschangesintheloadof tionsinresponsetimebyanorderofmagnitude,rangingfromabout5secondsto25seconds. Interestingly,theresponsetimesatdierentserverswerehighlycorrelated,alltendingtohave thatthepredominantfactordeterminingresponsetimeforthesesearchenginesislatency withinthenetwork,notloadorprocessingattheservers.wehaveveriedthatthepaths roughlythesameperformanceatagiventime.weinferfromthis(andotherdatacollected) 5 theselinkswillhaveasimilareectonallresponsetimes. Weperformedsimilarmeasurementsonasetofsearchengines.Weagainobservedvaria- 0 Serversthatareinacompetitivebusiness(e.g.,searching)haveaneconomicincentiveto Theseexperimentsillustratethatsomeserversandmetricsarenotamenabletoanycasting Number of Requests provisionadequatelytoavoidserverover-loading.auserwillprobablydojustaswellto selectoneoftheseserversarbitrarily. connectionbetweentheclientandtheserver(ethernetoratm).eachtransferconsistedof functionofserverload(measuredasthenumberofclientsbeingserved)andtypeofnetwork transferssuchasftp.wehavemeasuredserver-to-userthroughputusingftpclientsthat wehaveinstrumented10.figure5demonstrateshowserver-to-clientthroughputvariesasa 5.1.2Server-to-UserThroughput signicantamountsofdata(e.g.,webpageswithgraphics)andformoretraditionalbatch-style 10Theclientwasmodiedsothatitdidnotwritereceiveddatatothedisk;wedidthistoavoidhavingthedisk Server-to-userthroughputisausefulmetricforinteractiveapplicationsthatmusttransfer writebethebottleneckinthethroughputmeasurement. 13 Average Throughput at Client

15 FIGURE6FTPThroughputwithVaryingDistance 10 Internal [2 hops] Internal [3 hops] Internet [15 hops] Internet [16 hops] 8 10Mbytes. 6 4 canvaryconsiderably,bothacrossserversand(toalesserextent)onasingleserverover time.theserversthatareclosertotheclientgenerallyhavehigherthroughputandgreater doneforeachserver;thex-axisindicatestheindexofthetransfer,andthey-axisindicates Figure6demonstratesvariationinserver-to-clientthroughputbasedonthe(hop-count) 2 thethroughputtendstobe. distancebetweenserverandclient.tentransfersofapproximately100kbyteseachare themeasuredthroughput.theseresultsindicatethatthroughputbetweenserverandclient ServerLoad reasonable(inverse)proxyforthroughput;thefurthertheserverisfromtheclient,thelower variationinthroughputovertime.notethattheseresultssuggestthathopcountmaymakea Client Access Index applications.weobtainapproximatedataabouttheserverstatusbypost-processingthelogs becauseitismorepracticaltotrack,andmayprovidesucientinformationtobenetsome fromanhttpserver.specically,thelogsthatweaccessedcontainedarrivaltimeinformation forhttprequests.wecountedtheaggregatenumberofarrivalsintimeintervalsoflength Serverloadisamorecoarsemetricthanresponsetimeorthroughput.Weincludethismetric Tseconds,anddenedtheservertobe\loaded"ifthenumberofrequestsinaninterval exceededr.clearlytheparticularresultswillbesensitivetothechoiceoftandr,which Wehaveoverlaidresultsfromthreedierentdays Saturday,May11-Tuesday,May14, shouldbechosenbasedonservercapabilitiesandtimescaleofuctuationinload TheseresultsusetimeintervalT=60secondsandthresholdR=85requests.Wenote Figure7showstheloadstatusofanhttpserverwithinourdepartment( 14 Server Throughput (Mbps)

16 FIGURE7LoadStatusofHTTPServer Sat May Sun May Mon May Tue May ProcessorLoad Measurementofprocessorloadisausefulmetricforanapplicationthatisquitedierent fromtheserveraccessdescribedthusfar.forexample,aprocessmanagerresponsiblefor fourtracesexhibittime-of-dayvariations. distributingtasksontomachinescanviewasetofprocessorsasareplicatedservice,with thattheactivityisdistinctlydierentontheweekenddaysversustheweekdays.further,all servers(forgeandlennon)overatypical24hourperiod,withmeasurementstakenevery selectionofthebestmachinecorrespondingtotheleastloadedprocessor. network.figure8showstypicalresultdepictingtheload(asreportedbyrup)ontwocompute Wehaveexperimentedwithmeasuringprocessorloadforvariousmachinesonourcampus 12:00 A.M. 12:00 P.M. 12:00 P.M. Time 5minutes.Thisportionofthedatarunsfromabout2amto2am.Thesevaluesalsoshow Theprevioussubsectionindicatesthatthereareserversandmetricsthatcanbenetfrom 5.2MetricCollectionTechniques ofprocessors.inthisexamplebothmachineshavefourprocessorseach. thenight(roughly2am).toproperlycomparetheloadsonthevariousmachines,thedata time-of-daytrends,withoneoftheservers(forge)seeingsignicantactivityinthemiddleof anycasting.wenextconsiderhowtoecientlyandaccuratelymaintaindatabasesofthe shouldbenormalizedusingsomemeasureofprocessingcapability,suchasnumberandtype performanceinformationinanycastservers'database: metricvalues.wehaveidentiedfourpossibleapproachestomaintainingreplicatedserver or more accesses this minute?

17 FIGURE8ProcessorLoadonComputeServers Forge Lennon RemoteServerPerformanceProbing:Inthistechniqueprobingagentsareinchargeof ServerPush:Itmaybeadvantageousinsomecircumstancestohavethereplicated partsofthenetworkandtheserverthattheclientrequestwill\exercise".wehave asetofclients.theseprobesneedtobedesignedtomimic(asmuchaspossible)the experimentedwiththistechniqueinourimplementation.insection5.3wegivean exampleofourresultsandexaminetheissueofprobelocationandaccuracy. periodicallyqueryingthereplicatedserverstodeterminetheperformancethatwillbe experiencedifaclientweretorequestservice.eachprobingagentactsasaproxyfor 2 0 servertoupdateitsperformanceinformationonlywheninterestingchangesareobserved. theanycastservers.theprimaryadvantageofthistechniqueisscalability:itallowsthe serverssend(orpush)therelevantperformanceinformation(asmeasuredlocally)onto 2:00 A.M. 2:00 A.M. Time thatmaintaininformationabouttheserver.theanycastresolverscanjoinwell-known multicastgroupsforeachserverthattheyareinterestedin,allowingtheserversto Further,theupdateinformationcanbe(networklayer)multicasttoallanycastresolvers disseminateperformanceinformationwithoutknowingtheidentitiesoftheresolvers. Themulticastgroupsmaybeorganizedinotherways,forexamplewithonemulticast alternativesdierinthenumberofmulticastgroupsrequired,theeaseofmaintaining groupperanycastdomainname,oronemulticastgroupperanycastgroup.these themulticastgroupsandtheamountofnetworktracrequiredfordisseminatingserver information. Wehaveconsideredseveralvariationsonserverpushbasedonthetypeand/orfrequency ofinformationthatispushed.insection5.4wedescribeanexperimentthatconsiders pushinggoodnews,pushingbadnewsandperiodicallypushingcurrentstate. 16 Load

18 3.ProbingforLocally-MaintainedServerPerformance:Anotherpossibilityistohaveeach 4.UserExperience:Thelasttechniqueismotivatedbytheobservationthatuserscurrently niques.sinceprobesmerelyreadfromalocally-maintainedle,theymayrepresentless readablele.remoteprobinglocationscanthenreadtheinformationinthele(as sense,thisisahybridbetweentheprobing-for-performanceandtheserver-pushtech- opposedtoattemptingtoexercisetheserver)toobtainthedesiredinformation.ina replicatedservermaintainitsownlocallymonitoredperformancemetricsinaglobally serverperformance.theprimaryadvantageofthismethodisthattheinformationis time.collectinginformationaboutpastexperienceoersacoarsemethodofmaintaining collectedforfree;noadditionalburdenisplacedontheserverorthenetwork.the particularservertobeunreachable,thatserverislikelytobeavoidedforaperiodof makeserveraccessdecisionsbased,inpart,onpastexperience.thatis,ifonendsa ofaburdenontheserverthantheprobing-for-performanceapproach. costsassociatedwiththreeofthesetechniques. 5.3Example:ProxyProbingandUserExperience Wenextdescribeseveralexperimentsthatexamineperformanceandsystemoverhead amongclients.forexample,agatewayintoacampusmightmaintainserverperformance quantityandaccuracyoftheinformationcanbeincreasedbysharingofexperience Ideally,theprobeswhichcollectmetricinformationwouldberuninasfewlocationsaspossible informationbasedontheexperienceofallclientsonthecampus. andtheserver,thusfewerprobeswillcauselessoverheadformetricmonitoring.again,it yetstillprovideaccuratedatatobeusedinltering.eachprobeputsloadontothenetwork comparisonreecttheviewattheuser. isnotnecessarythattheabsolutevaluesofthemetricsbeprecise,onlythattherelative runateachuser,andthuswouldfollowapathfromtheusertotheserver.forscalability,it ispreferabletorunpath-dependentprobeson\proxies"thatprovidemetricdataforagroup fromalocationthatincludes(somepartof)thepath.attheextreme,theseprobescouldbe ofusers.theseproxiesmightbechosentocoverallhostsonasubnetwork,withinarouting domainorwithinageographicregion.asaproxycoversalargersetofhosts,thenumberof proxies(andthusnetworkloadduetoprobing)decreases,howeversodoestheaccuracyof theproxydatarelativetoanarbitraryuserinthegroup. Metricswhicharedependentonthepathfromtheusertotheserverwillneedtobeprobed maintainmetricdatawithnoadditionalloadonthenetworkorservers.weexpectmetric thesameasoccursintheproxyprobingmethod.thustheexperimentdescribednextalso providessomeinformationabouttheaccuracyofuserexperience. datathatismaintainedusingexperiencetobelessaccuratethandatathatismaintainedbya moreregularmeasurementprocess.therearetwosourcesfortheinaccuracy:(1)theservers performanceatanotherclient.wenotethatthissecondformofinaccuracyisessentially maynotbeaccessedregularly,and(2)theexperienceatoneclientmaynotaccuratelyreect Theexperienceofaclient(oragroupofclientsthatshareinformation)canbeusedto thefollowingexperimentalsetup.weranprobesonvemachines,threeonourlocalcampus Wehaveexploredthequestionofproxylocationandaccuracyoftheproxydatausing 17

19 local local local remote remote BestSecondThirdFourthFifthSixthSeventh theothertwowerechosentogetgoodgeographicaldistribution12.measurementsweretaken everythreeminutesforabout13hoursfrom23:00estto12:00estonthefollowingday. timetoasetofsevenserverslocatedaroundtheworld.fiveofthemarefreebsdservers; networkandtwoonthenetworkofanothercampus11.wemeasuredhttpserverresponse Inordertocomparetheaccuracyofdierentproxies,thetimesatwhichthereadings Table1:AccuracyofProxies betweenthetwocampuseswheretheproxieswerelocated). processesasquicklyaspossiblefromamachinerunningshellsoneveryproxy.moreelaborate theprobesareoccurringwithinsecondsofoneanother(modulodierencesinsystemclocks synchronizationiscertainlypossible,howeveraninspectionofthetimelogsindicatesthat inaccuracycausedbytakingthemeasurementsatdierentlocationsfromtheinaccuracy causedbytakingthemeasurementsatdierenttimes.)thisisaccomplishedbystartingthe aretakenmustberelativelysynchronized.(thatis,wewouldliketoisolatetheissueof thedatalescontainedtime-consistentprobemeasurementsacrossallproxies. probes.wepost-processedthedatalestondthemissesateachproxyandremovethe correspondingtimevaluesfromthedatalesfortheotherproxies.the\clean"versionsof thebaseandbyeachproxy,basedonthemeasurementdata.wethenrecordhowoftenthe othersasproxies.foreachmeasurementpoint,wedeterminethebestserverasselectedby proxiesagreeordisagreewiththebaseontheselectionofthebestserver.moreprecisely, Inthissetofdata,wedidndthattheproxiesoccasionallymissedregularlyscheduled wecounthowofteneachproxyagreeswiththebase(i.e.,selectsthesamebestserver),how oftentheproxyselectsthesecondbestbaseserver,andsoon.table1showsthepercentage oftimethateachproxypickseachserver. Toassesstheaccuracyoftheproxies,wedesignateonemachineasthe\base"andall withinourownlocalcampusnetwork.local1isthebasemachine.remote1andremote2 asmightbeobtainedwithoutanycastingsupport wouldpickthebestserver1/7thor localcampusagreemoreoftenandmoreaccuratelywiththebasethantheproxiesonthe remotecampus.eventheproxiesontheremotecampusgivefairlygoodresults,selectingthe bestorsecondbestserverabout50%ofthetime.(notethatarandomselectionofservers aretwomachinesondierentsubnetsoftheremotecampus'network.theproxiesonthe 14%ofthetime.) Themachineslabeledlocal1,local2andlocal3arethreehostsondierentsubnets minnie.cs.adfa.oz.au, amorediversesetofmachinelocations,howeverwewerelimitedinouraccesstowidelydispersedsystems. chinesorfaraway.wehavedividedtheanalysisaboveintotwoparts:onecontainingthe 12Theserversweprobedwere: 11Thetwocampusesareseparatedbyseveralstatesand3-4networkhops.Theseexperimentsshouldberunwith Theresultsareaectedbywhethertheserversarerelativelynearthemeasurementma- 18

20 local local local remote remote BestSecondThird Table2:AccuracyofProxies-NorthAmericanServers local local local remote remote BestSecondThirdFourth 3showtheresultsforthesetwogroupsofservers.Theadvantageofnearbyproxiesismore threenorthamericanserversandtheothercontainingthefourforeignservers.tables2and aslongastotheforeignservers,andhavemorecommonalityforthenearbyproxies. signicantforthenorthamericanservers;thepathsfromtheproxiestotheseserversarenot Alloftheproxiesdoquitewellfortheforeignservers,witheachproxyagreeingwiththe Table3:AccuracyofProxies-ForeignServers baseover50%ofthetime.theroutestotheforeignmachineshavemanyhopsincommon; forexample,allmachinessharethesamelast15hopstotheserverinindia. 5.4Example:ServerPush Todiscusstheperformanceoftheserverpushtechnique,wemustrstdenethealgorithm thattheserverwillusetodeterminewhatinformationtopushandwhen.ingeneral,we overheadoftheupdatingmechanism. \interesting"withsomeconstraintonthemaximumfrequencyofupdatessoastoboundthe wanttheservertopushstateinformationwheneverthestatehaschangedsucientlytobe TheservermeasuresitsstateovereachintervalI.Ifthestatechangesfromtheprevious thesamecriteria.wehaveadoptedthelinkstateupdatealgorithmusedinthearpanet[35] changebyatleastc,cisreducedbyr.(notethatwhencbecomes0,thestatewillbe measurementbyatleastc,thestateispushedandcisresettot.ifthestatedoesnot andexperimentedwiththeperformanceandoverheadwithavarietyofparameters.the andareductionfactorr.thealgorithmmaintainsacurrentthresholdc,initializedtot. updatealgorithmisparameterizedbyameasurementintervali,amaximumthresholdt Notethatthetaskofupdatinglinkstateinadistributedroutingenvironmenthasprecisely pushedandcwillberesettot.)thealgorithmwillsendupdatesatleasteveryt=rtime 19

21 FIGURE9PerformanceofServerPushAlgorithms Threshold = 10 Threshold = 20 Threshold = 30 Threshold = unitsandatmosteveryitimeunits. 40 surementquantityisthenumberofconnectionsinitiatedinthemeasurementinterval.inour experimentsweusei=1minute.weprocesshttpserverlogstodeterminethepushed 30 valuesforvaryingtandr. 20 R.Ateachoneminuteinterval,wecomparethevalueoftheserverloadbasedonthepushed 10 thedierencebetweentheestimateandthetruevalue,andplotafrequencydistribution.in informationtothetruevalueofloadextractedfromthelogs.werecordthemagnitudeof WehaveimplementedthisserverpushalgorithmforanHTTPserver,wherethemea- Figure9weusedR=5minutesandvaryingT;wealsocollecteddataforR=10minutes. InFigure9weexaminetheaccuracyoftheupdatingmechanismasafunctionofTand forvariousvaluesoftandr.thenumberofupdatemessagesgeneratedisameasure Table4givestheaveragevalueoftheinaccuracyandthenumberofupdatemessages, Error Magnitude summarizestheperformancedatagivenbythefrequencydistribution.thetradeoisclear:a oftheloadplacedonthenetworkbythemechanism;theaveragevalueoftheinaccuracy moreaccuratealgorithmalsoincursalargeroverhead.interestingly,increasingthemaximum intervalbetweenupdatesfrom5to10hasrelativelylittleeectontheaccuracyforaconstant networkand/ortheserver. themetricvalue,thedesiredaccuracyoftheinformation,andthecostofburdeningthe scaleonwhichthemetricvaries,thecomponents(server,networkpath,client)thatdetermine Whichtechniqueismostappropriatewilldependonanumberoffactors,includingthetime- updatethreshold. 5.5ComparisonofMetricCollectionTechniques 20 Error Frequency

22 ThresholdBetweenUpdatesErrorUpdates UpdateMax.IntervalAverageNumberof Table4:LoadandAccuracyofServerPush LoadModLoadNetPath NetServerServerExercisesAccuracy 379 Probing ServerPush ReadingServerLogPTpYes UserExperienceNoneNo 2PTpNoModerateYes TsYes Low No Moderate Table5:ComparisonofMetricCollectionTechniques (Seenoteintext) None YesLow/Varies High rstthreecolumnsaremeasuresofsystemoverhead.thenetloadcolumnrepresentsthe Pisthenumberofproxies,Tpistheperiodofproxyprobing,Tsistheperiodofserver numberofmessagesgeneratedperunittimetoobtainthemetricdatafromoneserver,where push.notethattheserverpushmessagesaremulticastratherthanunicast.theservermod ServerLoadcolumnexpresses(relatively)howmuchadditionalloadisplacedontheserverby columnindicateswhethertheservermustbemodiedtoallowthemetrictobecollected.the thecollectionofthemetricdata.thelasttwocolumnsareperformancemeasures,indicating Table5.5summarizesthefourtechniquesbasedonperformanceandcostdimensions.The whetherthemethodexercisesnetworkpath,and(relatively)howaccuratelythemethodis methodscanbemadetoexercisethenetworkpathbymeasuringhopcountorlatencyforthe abletomaintainthemetricsthatitcanevaluate.notethattheserverpushandserverlog metricupdatemessage. 21