UsingAutomaticMemoizationasa SoftwareEngineeringToolin

Transcription

1 UsingAutomaticMemoizationasa SoftwareEngineeringToolin ComputerScienceDepartment JamesMayeld Real-WorldAISystems UniversityofMarylandBaltimoreCounty Baltimore,MD MartyHall ScienceApplicationsInternationalCorp. 9841BrokenLandParkway,Suite118 Columbia,MD21046 UniversityofMarylandBaltimoreCounty ComputerScienceDepartment TimFinin Baltimore,MD April1994 detailedtreatment.thispaperdescribeshowautomaticmemoizationcanbemadeviableonalargescale. automationofmemoizationasapracticalsoftwareengineeringtoolforaisystemshasneverreceiveda cussedsincethesixtiesandareoftenusedasexamplesinintroductoryprogrammingtexts.however,the MemofunctionsandmemoizationarewellknownconceptsinAIprogramming.Theyhavebeendis- Abstract Itpointsoutadvantagesandusesofautomaticmemoizationnotpreviouslydescribed,describesthecom- 1

2 Track:EnablingTechnology. apubliclyavailablememoizationpackage(clamp)forthelispprogramminglanguage.experiencein applyingthesetechniquesinthedevelopmentofalargeplanningsystemarebrieydiscussed. ponentsofanautomaticmemoizationfacility,enumeratespotentialmemoizationfailures,andpresents Words:5869. Correspondence.Pleaseaddressallcorrespondenceto:JamesMayeld,ComputerScienceDepartment, Subtrack:SoftwareEngineering. UniversityofMarylandBaltimoreCounty,Baltimore,MD Voice: Fax:

3 theautomationofmemoizationasapracticalsoftwareengineeringtoolforaisystemshasneverreceived 1Introduction adetailedtreatment.thispaperdescribeshowautomaticmemoizationcanbemadeviableonalarge scale.itpointsoutadvantagesandusesofautomaticmemoizationnotpreviouslydescribed,itemizesthe MemofunctionsandmemoizationarewellknownconceptsinAIprogramming.Theyhavebeendiscussed componentsofanautomaticmemoizationfacility,enumeratespotentialmemoizationfailures,andpresents sincethesixtiesandareoftenusedasexamplesinintroductoryprogrammingtexts[16,13,14].however, thelisp1programminglanguage.wewillalsobrieydiscussourexperienceinapplyingthesetechniquesin apubliclyavailablememoizationpackage(clamp CommonLispAutomaticMemoizationPackage)for thedevelopmentofthesignaturemanagementsystem(sms)[18],adecisionsupportsystemthatprovides Itreferstothetabulationoftheresultsofasetofcalculationstoavoidrepeatingthosecalculations.Automaticmemoizationreferstoamethodbywhichanordinaryfunctioncanbechangedmechanicallyintoone thatmemoizesorcachesitsresults.weplacethedecisionaboutwhichfunctionstomemoizeinthehands oftheuser;thiscontrastswiththeapproachofmostowandcohen[11],whichtriestoautomaticallyinfer Theterm\memoization"isderivedfromtheterm\memofunction,"whichwascoinedbyDavidMichie[10]. submarinecrewswithsituationalawarenessandoperationaladviceinordertoreducedetectability. althoughasmostowandcohenpointout,thelatterapproachisnotapracticaltool. whichfunctionsshouldbememoized.thesetwoapproachestoautomaticmemoizationarenotincompatible, ofmostaisystems. theoutset,automaticmemoizationfacilitatesthekindofexploratoryprogrammingusedinthedevelopment beingassuredofreasonableperformance.byallowingtheprogrammertoavoidetheseeciencyconcernsat Automaticmemoizationallowstheprogrammertowritecertainfunctionswithoutregardtoeciency,while MemoizationisparticularlyaptforAIapplications.RapidprototypingisahallmarkofAIprogramming. functionalprogramming[5],andnaturallanguageparsing[12]havebeendescribedintheliterature.in allofthecasesthatwehavereviewed,theuseofmemoizationwaseitherbuiltintoinaspecialpurpose 1Throughoutthispaper,weusethenameLisptorefertothelanguageCommonLisp,asdescribedinSteele[15]. Theprincipleofmemoizationandexamplesofitsuseinareasasvariedaslogicprogramming[17,7,3], 3

4 computingengine(e.g.,forrule{baseddeduction,orcfgparsing),ortreatedinacursorywayasanexample softwareengineeringtechniqueunderhumancontrolhasneverreceivedadetailedtreatment.inthispaper, wereportonourexperienceindevelopingclamp,apracticalautomaticmemoizationpackageforcommon ratherthantakenseriouslyasapracticaltechnique.theautomationoffunctionmemoizationasapractical Lisp,anditsuseinalarge{scaleAIproject. Figure1.Thisalgorithmisusedtodeterminecoecientsofinterpolatedpolynomials.Thealgorithm,which isastandardoneinnumericalmethods,istakendirectlyfromnumericalmathematicsandcomputing[1]. Bymeansofillustration,considerthedivided{dierencealgorithmdd,whichisshowninpseudo{codein andhigh=10andanotherrecursivecallwithlow=1andhigh=9;eachofthesewillinturnmakearecursive computation.forexample,acalltoddwithlow=1andhigh=10willgenerateonerecursivecallwithlow=2 atree;thus,arecursivecalltoddwithparticularargumentsmayberepeatedmanytimesduringasingle ofinteresttoushere.thecalltreeforthedivideddierencealgorithmddformsanetworkratherthan Theapplicationitselfisnotparticularlyimportant;itistherun{timebehaviorofthealgorithmthatis thedivideddierencealgorithm,usingf(n)=cos(n)andtherstnnaturalnumbersasarguments.since callwithlow=2andhigh=9.memoizationofddcauseseachcalculationtobeperformedonlyonceand storedinatable;thereafter,eachrepeatedrecursivecallisreplacedbyatablelookupoftheappropriate value. ofpointsiscalculatedmorethanonce,andthereare(n2+n)=2subsequences.subsequentinvocationstake (2n)timecomplexity.Aftermemoization,therstinvocationrequires(n2)time,sincenosubsequence asinglefunctioncallonnpointsgeneratestworecursivecallsonn?1points,theunmemoizedversionhas Figure2comparestheperformanceofmemoizedandunmemoizedversionsofaLispimplementationof near{constanttime. istouseanautomaticmemoizationpackagetoconverttheelegantbutinecientrecursivealgorithmintoa recursivedenitionswhicharesimplyunusableformostrealproblems.theconventionalresponsetothis situationistomanuallyrewritethealgorithminadynamicprogrammingstyle.ofcourse,suchmanual rewritingofthecodetakeseortandinvolvestheriskofintroducingnewbugs.anattractivealternative Thisalgorithmfordivideddierenceistypicalofalargeclassofalgorithmswhichhaveveryelegant 4

5 ;divided-differencealgorithm dd(points:array, begin iflow=highthen low:array-index, high:array-index, fn:function) end elsereturn(dd(points,low+1,high,fn)- returnfn(points[low]) (points[high]-points[low]) dd(points,low,high-1,fn))/ Figure1:Thedivideddierencealgorithmfordeterminingcoecientsofinterpolatedpolynomialscanbe elegantlywrittenusingrecursionandmadeecientthroughtheuseofautomaticmemoization nunmemoizedmemoizedmemoized (rstrun)(subsequentruns) Centuries isreducedfrom(2n)to(n2)forinitialrunsandtonear{constanttimeonsubsequentones. Figure2:ThistableshowsthebenetsofmemoizationonaLispimplementationofdd.Thetimecomplexity

6 rewritingcodebyhand.sectionfourdescribesthegeneralcomponentsthatshouldbepresentinany describedintheliterature.sectionthreecomparestheuseofautomaticmemoizationtothealternativeof realapplication. usefulone.thisisattractive,ofcourse,onlyifsuchapackagecanaddressthepracticalproblemsfacedin variouswaystoaddressthem.sectionsixdescribesourexperienceinusingautomaticmemoizationinin automaticmemoizationfacility.sectionvepresentsproblemsinherentintheuseofmemoizationandthe Inthenextsectionwedescribethesomeoftheaspectsandusesofautomaticmemoizationnotpreviously Therearefourmainusesofautomaticmemoization.Twooftheseinvolvetheavoidanceofredundant 2UsesofMemoization thedevelopmentofsms alargedecisionsupportsystem. calculation,rst,withinasinglefunctioninvocation,andsecond,acrossinvocations.thethirduseof arenotconjecturedusesbutoneswhichwefoundtobeeectiveinmanysituationsinalarge,real{world AIapplication.Wediscusseachoftheseusesinmoredetailinthefollowingsubsections. 2.1RepetitionwithinaFunctionCall automaticmemoizationisasapre{calculationtool,whilethefourthisasatimingandprolingtool.these call.inthedivideddierenceexamplepresentedabove,thereweremanyrepeatedrecursivecallswithina Themostcommonuseofmemoizationistoavoidtherepetitionofsub{calculationswithinasinglefunction singlefunctioninvocation.thistypeofrepetitioniscommon.forexample,asimplerecursivebacktracking thathasthesamebehaviorasadynamicprogrammingalgorithm;ratherthandeterminingtheproper parsermayparsethesameconstituentmanytimesduringasingleparse;thusitsperformanceispoor. orderinwhichtoconstructsubpieces,memoizationofasimplesolutioncantypicallyachievethesame algorithm[4]throughtheapplicationofmemoization. Norvig[12]hasshownthatsuchanalgorithmcanobtaintheperformanceofchartparsing[8]orofEarley's performance[2].again,throughautomaticmemoizationonecan,inmanycases,achievethebenetsofa Thus,wecanviewmemoizationasatechniquetoautomaticallyconvertarecursivealgorithmintoone 6

7 Inateamprogrammingenvironmentdierentsectionsofasystem,writtenbydierentprogrammers,may 2.2RepetitionoverTime toholdtheintermediateresults. dynamicprogrammingapproachwithoutrewritingthecontrolstructureorintroducingnewdatastructures timesthatmakeuseofsomethesamepieces.inthesecases,thereisnocentralroutinewhichcouldmanage accessthesamefunction.alternatively,inaninteractivesystemtheusermayinvokecalculationsatdierent insuchcasesistohavetheroutineinquestionmanageitsowndatastructurestocachepreviousresults. thecallingsequencetoavoidrepetitionofthecalculations.theonlyalternativetoautomaticmemoization Theprecedingsubsectionsshowedthatmemoizationcaneliminatetherepeatedinvocationofexpensive 2.3Persistence calculations.thesetwoapplicationsofmemoizationareusefulwhenitisfeasibletoperformtherst invocationofafunctionatrun{time.memoizationisalsousefulwheneventherstinvocationistoo expensivetoperformatrun{time. purposedatale,andstoringinittheresultsofano{lineexecutionoftheexpensiveroutine.then, thefunctioninquestionismodiedtoaccessthatle.automaticmemoizationprovidesamethodto memoizationeliminatestheneedfortheprogrammertoknowwhichrangesofvaluesarestoredinthedata pre{calculateafunctionwithouttheoverheadofahand{craftedsolution.insuchsituationsautomatic Useoffunctionsthataretooexpensivetocalculateatrun{timeisusuallydonebybuildingaspecial andthenruno{lineonthecasesofinterest.thecontentsofthehashtablearethensavedtodisk.the le,andwhichmustbecalculated.toachievepersistenceofanexpensivefunction,thefunctionismemoized functionaredeterminedautomatically.thatis,theprogrammerdoesnothavetospecifytherangeof allpossibleinputstothememoizedfunctionarepre{calculated.second,theinputvaluestothememoized savedleislaterusedtoseedthehashtableforthefunctionwhenitisreloaded. calculateafunction.first,itallowstheeliminationoffunctionsfromtherun{timesystemincaseswhere Therearetwoadditionaladvantagesofthistypeofmemoizationbeyondprovidingtheabilitytopre{ 7

8 possibleinputstothememoizedfunction.infact,thissolutionworkseveniftheprogrammerhasnoidea whichinputvalueswillbeused. 2.4TimingandProling systemsprovideaprolingfacilitywherebytheusercanseethetimethatatop{levelfunctionspendsin cantakethirtytimeslongerthananunmeteredrun.thisdoesnotincludethetimerequiredtoload Finally,automaticmemoizationcanalsousedasaprolingandtimingtool.Manyprogramminglanguage variouslower{levelroutines.thisisimportantfordirectingoptimizationeorts.however,theseprolers generallyrequiresignicantoverhead.forexample,afully{meteredlisprunonasymbolicslispmachine themeteringsystem.theexpenseofmeteringisworththeeortforimportantcases,andisavaluable softwareengineeringtool.insmallercases,however,automaticmemoizationprovidesaquickbutrough methodfordeterminingwhichroutinestooptimize.ratherthanrunningthefullymeteredsystem,users interactivelymemoizecertainfunctions,thenrepeattheoriginaltestcasetwice.ifthetimingforthesecond caseimprovesonlyby,forexample,vepercent,thenforthattestcase,noamountofoptimizationinthe routinesinquestionwillprovidemorethanavepercentspeedup.ifontheotherhandagreatspeedupis seen,thenthememoizedfunctionsaregoodcandidatesforoptimization. memoization,dynamicprogramming,anddevelopmentofanewalgorithm.first,memoizationneednotbe automated.memoizingaroutinebyhandcouldconceivablyresultinminoreciencygainsoverautomated Therearethreealternativestoautomaticmemoizationforeliminatingrepeatedcalculation:hand{crafted 3AlternativestoAutomaticMemoization memoization.second,insomecasesanorderingcanbefoundforthecalculationstobeperformedsuch ofcomputerprogramming[9],knuthpresentsastraightforwardmethodforcalculatingdivideddierences intheproperordertogetthesameperformanceastherstinvocationofthememoizedversionpresented intheintroduction.finally,anewalgorithmforagiventaskcanbesoughtthatdoesnotrequirerepeated calculations. thatfullmemoizationisnotneeded.forexample,involume2(seminumericalalgorithms)ofhistheart 8

9 memoizationovertheotherapproachesdescribedabove.theyfallintothreecategories:qualityofsolution, themajorbenetofthedevelopmentofanewalgorithmisasavingsinrepeatedcalculations,automatic easeofuse,andadditionalusesofmemoization. memoizationofanexistingalgorithmhasseveraladvantages.theseadvantagesalsorecommendautomatic Automaticmemoizationisnotasubstituteforndingtheproperalgorithmforatask.However,when 3.1Qualityofsolution does.thislastpointisespeciallyimportantinthedevelopmentoflarge,complexsystemwherethereis notrisktheintroductionofbugsintothefunctiontothesamedegreethatdevelopingnewcodeforthetask functiontobememoizedhasalreadybeenwrittenanddebugged,theuseofautomaticmemoizationdoes eciencyimprovementdoesnotappearinthebodyofthefunctionbeingimproved.furthermore,ifthe Automaticmemoizationusuallyleadstoshort,clearimplementations,becausethecodetoimplementthe anaturalreluctancetochangeroutinesthathavealreadybeentestedandveried,especiallyifthatwill requirechangesinmultipleplacesinthecode.furthermore,becauseitissimpletoswitchbackandforth Inmostlanguagesautomaticmemoizationcanbeimplementedsothatitissimpletomemoizeandunmemoize 3.2Easeofuse functions.noneofthealternativestoautomaticmemoizationcanboastsuchalightloadontheprogrammer. betweenthememoizedandunmemoizedversions,itiseasytocomparetheperformancesofthetwoversions. thedesignofsuchapplicationstendstochangerapidlyandfrequently. maintainingnewcode.thiseaseofuseisespeciallyimportantinarticialintelligenceapplications,because 3.3Additionaluses Again,thiseaseofusedependsinpartonavoidingtherequirementofwriting,debuggingandeventually not,ingeneral,providetheotherbenetsofautomaticmemoization,i.e.persistenceofcachedvaluesand Whilethealternativestoautomaticmemoizationmentionedaboveeliminaterepeatedcalculations,theydo usefulnessasatimingandprolingtool.itwouldbepossible,ofcoursetobuildinsomeofthesefeatures 9

10 onlyonce inthegeneralmemoizationfacility. tothedevelopmentofthesmssystem. suchasapersistentcachemechanism.however,theautomaticmemoizationapproachrequiresustodothis buildingrealsystems.hallandmayeld[6]describesomeoftheseadvantagesinmoredetailastheyapplied Theresultofthesebenetsisthatautomaticmemoizationoerssignicantpracticaladvantagesin BasedonourexperienceindevelopingandusingCLAMP,wehaveidentiedcharacteristicsthatanyautomaticmemoizationfacilityshouldhave.Automatedmemoizationofsomesortcanbeimplementedin mostlanguages.languagesthatprovidetheuserwithaninteractiverun{timeenvironment(suchaslisp, DylanandML)musthaveeitherrun{timefunctionnameresolutionorapatchablecodesegmenttoallow automatedmemoization.languagesthatarestrictlycompiled(suchasmostimplementationsofc)must 4ComponentsofanAutomaticMemoizationFacility 4.1MemoizationControl beaugmentedwithsomesortofpreprocessortoallowautomatedmemoization. First,thesystemmustprovidetheuserwithavarietyofmethodstoeasilymemoizeandunmemoizefunctions. functions,cachecontrolfunctions,andstatisticsreportingfunctions.wewilldiscusseachinturn. Threetypesofcontrolfunctionsmakeupausefulmemoizationpackage:memoizationandunmemoization afunction,unmemoizeallfunctions),andalso,ifpossible,temporarymemoizationcommands(e.g.treat evaluatingaparticularexpression). afunctionasmemoizedwhileevaluatingaparticularexpression,treatafunctionasunmemoizedwhile Thesemethodsshouldallowbothabsolutecontrolovermemoization(e.g.memoizeafunction,unmemoize experimentationshowsnobenetsfromdoingso.2 hisorherprogram.itisnotalwaysobviouswhatimpactmemoizingafunctionwillhave.forexample,at rstglance,thehailstonefunctionshowninfigure3lookstobeagoodcandidateformemoization,but 2Ofcourse,somecarefulreasoningwillalsoleadaprogrammertothesameconclusion,butnotallprogrammerswillhave Itisimportanttoallowtheprogrammertoexperimentwiththeeectsofmemoizationinthecontextof 10

11 (defunhailstone(n) (+1(cond((=n1)0) ((evenpn)(hailstone(/n2))) 4.2CacheControl Figure3:TheHailstonefunctionappearstobeagoodcandidateformemoization,butitisnot. (t(hailstone(+1(*3n))))))) Second,thesystemmustprovidemethodstocontrolindividualmemoizationcaches,bothforasinglesession andacrosssessions.mostimportantistheabilitytoexplicitlyclearacache,therebyforcingallfunction savedtodisk,andinalatersession,toinstructthatsuchasavedcachebereadfromdisk.easeofuse returnvaluestobere{calculated.persistenceisprovidedbyallowingtheusertoinstructthatacachebe concernsdictatethatthesemethodsbeactivatedbyreferringtothenameofthememoizedfunction,and nottothenameofadiskleortosomeotherinternaldatastructure. thenumberoftimesthatsuchinvocationsresultinthereturnofacachedvalue.theusershouldalsobe functionisrunning.thesestatisticsshouldincludethenumberoftimesamemoizedfunctioniscalled,and 4.3DataControl allowedtoresetthesestatisticsinpreparationforanewrun. Finally,thememoizationpackageshouldprovideroutinesthatreportstatisticscollectedwhileamemoized 5MemoizationFailures thetimeandexperiencetodothistypeofanalysis. Amajoradvantageofautomaticmemoizationisitstransparency.However,anoverly{transparentviewcan (notablybymostowandcohen[11]),mosthavenot.instead,welearnedthemthehardwayinusingthe leadtoproblems.whilesomeaspectsofthesememoizationfailureshavebeendiscussedintheliterature 11

12 severalyearsindevelopingthesmssystem. useditisthattheuseofatechniquethatcannotguaranteecorrectresultsafteritsapplicationisoutof evolvingclampsystemthroughtheexperiencesofaiprogrammersusingthefacilityoverthecourseof thequestion.thiscomplaintisunfounded.asacounterexample,atechniquethatiswidely{usedbyc programmersisfunctionin{liningthroughmacroexpansion.suchmacroexpansionisnotguaranteedto producethecorrectresultsinallcases.however,becausetheprogrammercontrolswhenmacrosareapplied, Themostcommoncriticismofautomaticmemoizationthatwehearfromprogrammerswhohavenot 5.1CacheConsistency thetechniquecanbeusedprotably.itisforthisreasonthatweadvocateprogrammercontroloverthe Memoizationisusedtoavoidrecalculationwhenafunctionisinvokedwithargumentsithasseenpreviously.Directchangestothefunctionshouldresultinthememoizationcachebeingushedautomatically,as ofallfunctionsinitscalltree,andrequirethatthecachebeushedwhenanyofthoseentriesismodied. Thiscouldnotbedoneautomaticallyatrun{timewithouttheuseofspecialdatastructuresintheunmemoizedfunctions;thiscapabilityisunlikelytobeadoptedinmostprograms.Inmanycases,aproperuseof todetectisthealterationofoneormoreofthesub{functionsthatthememoizedfunctioncalls.thereare severalwaystoalleviatethisproblem.thebestmethodwouldbeforeachmemoizedfunctiontokeepalist haveencounteredinmakingdecisionsaboutwhichfunctionstomemoize. selectionoffunctionsformemoization.inthefollowingsubsections,wedescribethepotentialpitfallswe presumablyitwillthencontainincorrectvalues.thissituationisrelativelyeasytodetect.moredicult goalwillbecompromised. fortheprogrammertoadopt.ifusingmemoizationrequiresinvasivechangestounmemoizedroutines,this Thisisatrade{o.Oneofthegoalsofanautomaticmemoizationfacilityistoprovidetoolsthatareeasy thecurrentlymemoizedfunctions,awarningtocheckthelistwheneverchangesaremadeisoftensucient. modularitywillindicatehowfarchangesarelikelytopropagate.sincetheuseralwayshasaccesstoalistof Run{timechangestofunctionsusuallyoccurduringdevelopment.Ifcachesarenotsavedtodisk,the Howeverthisproblemisaddressedwithinasinglesession,itisusuallynotaproblemacrosssessions. 12

13 memoizedroutineswillonceagaincalculatecorrectvaluesthenexttimethesystemisloaded. nodierentfromnormaldatales,andthesameproblemsofoutdatedversionsapply.nofullyautomated solutionwillguaranteeconsistency,shortofasystembuilderthatforcesrecalculationofallsavedvalues whenevertherearechangesinanycodeonwhichthefunctiondepends.thisisnotanunreasonableoption duringdevelopment,sincethememoizationfacilitymakesiteasytosaveentries.o{linecalculationscan Cachesthataresavedtodiskpresentamuchmoreseriousproblem.Insomesenses,suchcachesare beperformedatregularintervals.however,thereisstillanincentivetolimitthesecalculations,sincetheir recalculatememoizedvalues.first,theprogrammerspeciesapercentageoftheentriesthataretobe theoriginalfunctioneventhoughitsargumentshavebeenseenbefore.ineachcase,theserecalculatedvalues recalculatedonloadingthehashtable,and/orapercentageoftimeswherethememoizedfunctionwillinvoke time{consumingnatureiswhatledtosavingthetablestodiskintherstplace. arecomparedtotheoriginalones;awarningisgiveniftheresultsarenotidentical.load{timerecalculation Onewaytolimitthelikelihoodofinadvertentuseofobsoletevaluessemi{automaticallyistoperiodically maynotbepossible,ofcourse,sincealltheroutinesneededforacalculationmaynotbeavailablewhenthe specication,ifthesystemiscountingonagivenlevelofperformance.evenifbothtechniquesareused, theydonotguaranteethatallobsoletevalueswillbefound.thus,thesetechniquesshouldbeusedasa thanasareplacementforthem. functionisdened.similarly,run{timerecalculationmaynotbedesirable,evenwithaverylowpercentage 5.2Non-Functions Memoizationonlyworksfortruefunctions,notprocedures.Thatis,ifafunction'sresultisnotcompletely complementtoothersoftwareengineeringtechniques(suchasexplicitmaintenanceofdependencies),rather programmingstylethroughoutthesystem. numberoffunctionsthatcanbememoizedsuccessfullywillbeincreasedbyencouragingtheuseofafunctional anddeterministicallyspeciedbyitsinputparameters,usingmemoizationwillgiveincorrectresults.the 13

14 5.3ModicationofResults whenrstwritten,butwillmakesubsequentmodicationsdicult.however,insomecasestheycanleadto iscalled.thus,memoizedfunctionscanreturnresultsthatsharestructures,eveniftheunmemoizedversion Inherentintheideaofmemoizationisthatdataisstored,ratherthancalculatedaneweachtimeafunction alwayscreatesnewstructures.evenwithoutmemoization,operationsthatmodifyfunctionresultsare eciencygains;withcare,programmerscanusethemtospeedupthefunctionsthatcanreallybenetfrom theiruse.unfortunately,thetransparentviewofmemoizationbreaksdownwhenusedwithsuchroutines. dangerousfromasoftwareengineeringperspective.acommonproblemisthatsuchroutineswillworkne functionnormalized-data,whichdestructivelyremovesthemaximumandminimumentriesfromthelist eachsubsequentretrievalofsupposedlyidenticaldatavaluesmightinfactreceiveashorterlist.avoiding beforereturningit.priortomemoization,thismightbeperfectlysafe.aftermemoizingraw-data,however, thisproblemnotonlyrequirestheuserofmemoizationtoknowhowthefunctiontobememoizedworks, Forinstance,supposethatfunctionRaw-Datareturnsanewly{createdlistofnumbers.Itiscalledbythe functionsthattrulyneedtousethem. 5.4CompilerOptimizationofRecursiveCalls thestandardsonwhenmodifyingoperationsareallowed,andtorequireincreaseddocumentationforthose butalsohowitwillbeusedbyotherfunctions.thisisoftenadiculttask;aneasierapproachistotighten memoizationprocess.toavoidthisproblem,thecompilermustbeinstructednottoeliminatetail{recursion tioneliminatestherecursivefunctioncall,butnottheworkrequiredtocalculatethefunction'sresult.since memoizationreliesonexplicitfunctioncallstoactivateitstable{lookup,suchoptimizationwillbypassthe inmemoizedroutines.compilersthatdooptimizetail{recursionusuallyprovideanappropriatecompiler Somecompilerswilloptimizethecodetheyoutputbyconvertingtail{recursiontoiteration.Suchoptimiza- atrun{time.suchcompilerswilloftenbypassthenameresolutionprocessfordirectrecursion.when memoizationisimplementedbybindingthememoizedfunctiontotheoriginalfunctionname,thisonce directive,forusebythememoizationmachinery. Amoresubtleoptimizationissometimesmadebycompilersforlanguagesthatresolvefunctionnames 14

15 againresultsinacircumventionofthememoizationtable{lookup.thefunctionwillstillreturncorrect results,butthecomputationsavingsprovidedbymemoizationwillbelost.itislesscommonforacompiler togivetheuserexplicitcontroloverthiskindofoptimization.3 theresultsoftheoptimized{awayrecursivecallswillnotbecached,theresultsofthetop{levelcallswillbe cached.inthetermsoftheusecategoriesdescribedinsection2,thebenetsofrepetitionwithinafunction callarelostbutthoseduetorepetitionovertimearenot. Notethatthisproblemcaneliminatesome,butnotall,oftheadvantagesofmemoization.Although 5.5RecognizingCallEquivalence default.iffunctionfooisdenedas: Memoizationisperformedbydoinganexactmatchontheargumentlist,usingtheLispfunctionequalby andismemoized,allofthefollowingwillbetreatedasdistinct,eventhoughtheparametershaveidentical valuesinallcases: (defunfoo(&key(bar2)(baz3)...) (Foo) (Foo:Bar2) (Foo:Bar2:Baz3) that,forinstance,2isnotequalto2.0,and isnotequalto ,eventhoughthefunction Similarly,onecanhavecounterintuitiveresultswhentheargumentsareoatingpointnumbers,forgetting (Foo:Baz3) maytreatthemasidentical.thesolutionadoptedbythesmsprogramistointroduce\wrapper"functions (Foo:Baz3:Bar2) thatthecommercialcommonlispsystemwewereusingfailedtofollowthatdirective. form,thenpassthemontoaninternalfunctionthattakesonlyrequiredargumentsinthestandardformat. Itisthisinternalfunctionthatisthenmemoized. thattakekeywordarguments,oatingpointnumbers,etc.,canonicalizetheargumentsintosomecommon 3CommonLispcompilershavethenotinlinedirectiveforthis,althoughwediscoveredinthecourseofdevelopingCLAMP 15

16 5.6CacheValueRepresentation Inthecurrentsystem,theroutinesthatsavedatatodiskdosobyprintingtherepresentationoftheobject usingformat,directingtheoutputstreamtoale.thismeansthatlispobjectswhoseprintrepresentation cannotbeparsedbyreadcannotbesavedtodisk.someobjectssuchasclosinstancesandstructures allowthedenitionofacustomprintfunction,andthiscansometimesbeusedtosavethemtodisk.but thisisnotageneralmechanism,andspecial{purposecodewillneedtobewritteninthosecases. awarenessandoperationaladvicetohelptheshipreduceitsoveralldetectability.ithasbeendeveloped TheSignatureManagementSystem(SMS)isadecisionaidforsubmarinecrewsthatprovidessituational 6ExperienceandEvaluation CommonLisp,withtheremainderinC;itrunsonUNIXplatforms. andtwouniversitiesonthedevelopmentteam.thesystemcombinesavarietyofrepresentationsincluding overthelastveyearsunderarpafundingatthejohnshopkinsuniversityappliedphysicslaboratory (JHU/APL).OutsideofAPL,teammembershavecomeprimarilyfromindustry,witheightcorporations frames/objects,productionrules,mathematicalmodels,andproceduralcode.about75%iswrittenin memoization.allfourusesdescribedinsection2wereusedextensivelybymultipleprogrammers. threedierentcompanies.\permanent"memoization(i.e.usesotherthanproling)remainsinthereleasedsystemin25places.however,useasarst{cutprolingtoolwasoneofthemostcommonusesof TheautomaticmemoizationsystemwasusedintheSMSprogrambyatleastsixLispdevelopersfrom aftervarioususesofmemoizationwereputintoeect.ignoringthebenetswhentheuserasksforthesame 6.1SMSMagneticsModule memoizationonthersttimeinvocationofthetop{leveldisplay(whichisshowninfigure5).timesarein displayatdierenttimes(whichisinfactquitecommon),figure4givesasummaryofthetimebenetsof seconds,andareconservativeapproximations.similarresultswereobtainedwithothermodules. Figure4givestimingstatisticsforamagneticsmoduleusedintheSignatureManagementSystem,timed 16

17 aspect unmemoizedoriginal +conventionaloptimization +repetitionsovertime +dynamicprogrammingtime(sec)speedup +savedlookuptables , magneticsmoduleusedinthesignaturemanagementsystem. 6.2SMSDetectabilityPlanningDisplay Figure4:Theseguresshowthecumulativeeectsofthedierentaspectsofautomaticmemoizationona (theta)indicatingheading(0correspondingtoduenorth),andtheradius(r)correspondingtospeed.each detectionofthesubmarineforvariouschoicesofheadingandspeed,drawnonapolarplotwiththeangle estimatetheoverallcontributionofmemoizationtothesystem.forinstance,figure5showsadisplay usedasanaidtoplanningsubmarineoperationsinthesmssystem.itshowsthepredictedprobabilityof GiventhediverseusesofmemoizationbyvariousprogrammersontheSMSprogram,weattemptedto (r,theta)pair(arc)inthedisplayiscodedwithacolorindicatingthecumulativeprobabilityofdetectionfor methodsorsignatures.theuserfrequentlyisinterestedinthecontributionofindividualsignaturecomponentstothiscomposite.sincetheprobabilityofdetectionofeachcomponentismemoizedbeforeitis combinedintothecomposite,anycomponentcorrespondingtoapointonthedisplaycanberetrievedalmost Thisdisplayisusedasahigh{leveltoolinplanning,andthusincludeshighlysummarizedinformation. thesubifitweretooperateattheindicatedcourseandspeed. Itpresentsasinglenumberforprobabilityofdetectionwhichisacompositeofallthepotentialdetection computation. instantly.takingadvantageofthis,thedisplayoffigure6canbemaintainedwithvirtuallynoadditional speedandcourseforthepointunderthemouseiscalculated.then,theindividualcomponentsarecalculated, withtheirrelativevaluesshowninthebarcharts.duetotheeectsofmemoization,thecomponentvalues Whenevertheusermovesthemouseoverthecompositedetectabilitydisplay(Figure5),thecorresponding 17

18 probabilityofdetectionforthesubifitweretooperateatthecourseandspeed. indicatingheading,andtheradiuscorrespondingtospeed.eacharciscolorcodedtoindicatethecumulative ofdetectionofthesubmarineforvariouschoicesofheadingandspeed,drawnonapolarplotwiththeangle Figure5:Thetop{leveldisplayfromtheSMSsubmarineplanningsystemshowsthepredictedprobability 18

19 Theresults,giveninFigure7showa631ximprovementinspeed,anda4,822ximprovementintheamount canbecalculatedandgraphedasquicklyastheusercanmovethemouse. oftemporarymemory(garbage)allocated.benchmarksarenotoriouslymisleading,andinmanyplacesthe codewouldhavebeenwrittendramaticallydierentlyifmemoizationhadnotbeenavailable.nevertheless, theresultsareilluminating,especiallysincetheyrepresentimprovementsovertheoriginalbaselinesystem. Thesystemwasrunfromthisdisplayinthedefaultmodeandthenwithallmemoizationturnedo. Sincethecomputationofthissummarydisplayrepresentsthenal,high{levelcomputationoftheentire systemitisareasonablewaytomeasurethecontributionoftheuseofautomaticmemoization. 7Conclusions Automaticmemoizationisapowerfultoolthatallowsmanysimplebutinecientalgorithmstobemade theprevalenceofarapidprototypingapproachinsuchprojects. faroutweighthepotentialpitfallsofautomaticmemoizationinarticialintelligenceapplications,becauseof toamemoizedfunction,andtheabilitytoperformtimingandprolingstudiesrapidly.theseadvantages advantagestothearticialintelligenceprogrammer.theseadvantagesincludetheabilitytoaddpersistence usefulinpractice.beyondthisbasicadvantagethough,automaticmemoizationprovidesothersignicant Acknowledgements.ThisworkwassupportedinpartbytheAdvancedResearchProjectsAgencyunder ThinkingMachines,andispartoftheCMUAICD{ROM. SourcecodefortheCLAMPsystemisavailableviaanonymousFTP(ftp://ftp.cs.umbc.edu)orby (AAICorporation),J.PaulMcNamee(AAICorporation),PeterNorvig(SunMicrosystems),andDavidJ. Scheerer(TheJohnsHopkinsAppliedPhysicsLaboratory)fortheirhelpfulcomments,bothontheLISP theuseofthedivided--differenceexample.19 JHU/APLsubcontract605089{L.TheauthorsthankV.J.Benokraitis(AAICorporation),LienT.Duong implementationandonearlierversionsofthispaper.thankstojohnaspinall(symbolics)forsuggesting

20 signatureswhichmakeuptheoverallprobabilityofdetection.thiscomponentialanalysiswastrivialto Figure6:IntheSMSsystem,abarchartshowstherelativestrengthofeachoftheseveralcomponent recoverasasideeectofthedecisiontomemoizethecomputationoftheoverallprobabilityofdetection. Figure7:DramaticimprovementsinexecutiontimeandconsingwereobtainedintheoverallSMSsystem. unmemoized sec.2,969,392,724 memoized4.06sec. versiontimebytesconsed 615,784 20

21 References [2]ThomasH.Cormen,CharlesE.Leiserson,andRonaldL.Rivest.IntroductiontoAlgorithms.MIT [1]WardCheneyandDavidKincaid.NumericalMathematicsandComputing.Brooks/Cole,1980. [4]J.Earley.Anecientcontext-freeparsingalgorithm.CommunicationsoftheAssociationforComputing [3]S.W.Dietrich.Extensiontables:Memorelationsinlogicprogramming.InFourthInternationalSymposiumonLogicProgramming,pages264{273,1987. PressandMcGrawHill,1990. [5]AnthonyJ.FieldandPeterG.Harrison.FunctionalProgramming.Addison-Wesley,1988. Machinery,6(2):451{455,1970. [6]MartyHallandJamesMayeld.ImprovingtheperformanceofAIsoftware:Payosandpitfallsinusing [7]B.Homann.Termrewritingwithsharingandmemoization.InH.KirchnerandG.Levi,editors, automaticmemoization.inproceedingsofthesixthinternationalsymposiumonarticialintelligence, pages178{184.megabyte,september1993. [8]M.Kay.Algorithmschemataanddatastructuresinsyntacticprocessing.InProceedingsoftheSymposiumonTextProcessing.NobelAcademy,1980. Berlin,Heidelberg,1992. AlgebraicandLogicProgramming:Proc.oftheThirdInternationalConference,pages128{142.Springer, [9]DonaldE.Knuth.TheArtofComputerProgramming,volume2.Addison-Wesley,1969. [10]DonaldMichie.\memo"functionsandmachinelearning.Nature,218(1):19{22,April1968. [11]JackMostowandDonaldCohen.Automatingprogramspeedupbydecidingwhattocache.InProceedingsoftheNinthInternationalJointConferenceonArticialIntelligence,pages165{172.MorgaputationalLinguistics,17(1):91{98, [12]PeterNorvig.Techniquesforautomaticmemoizationwithapplicationstocontext-freeparsing.Com- KaufmannPublishers,Inc.,1985.

22 [13]PeterNorvig.ParadigmsofAIProgramming:CaseStudiesinCommonLISP.MorganKaufmann, [16]GerrySussmanandHalAbelson.TheStructureandInterpretationofComputerPrograms.MITPress, [15]GuyL.Jr.Steele.CommonLisp:TheLanguage.DigitalPress,secondedition,1990. [14]F.C.N.PereiraandS.M.Shieber.PrologandNatural-LanguageAnalysis.csli,Stanford,CA, [17]DavidS.Warren.Memoingforlogicprograms.CommunicationsoftheACM,35(3):93{111,1992. [18]DavidWenstrand,H.LeeDantzler,MartyHall,DavidScheerer,CharlesSinex,andDavidZaret.A DARPAAssociateTechnologySymposium,June1991. multipleknowledgebaseapproachtosubmarinestealthmonitoringandplanning.inproceedingsofthe 22