15.0. Percent Exceptions

Transcription

1 WhyCOTSSoftwareIncreasesSecurityRisks GaryMcGraw ReliableSoftwareTechnologies 21515RidgetopCircle,Suite250,Sterling,VA20166 phone:(703) ,fax:(703) Abstract UnderstandingtherisksinherentinusingCOTS softwareisimportantbecauseinformationsystemstodayarebeingbuiltfromevergreateramountsofreused andpre-packagedcode.securityanalysisofcomplex softwaresystemshasalwaysbeenaseriouschallenge withmanyopenresearchissues.unfortunately,cots softwareservesonlytocomplicatematters.often, codethatisacquiredfromavendorisdeliveredinexecutableformwithnosourcecode,makingsometraditionalanalysesimpossible.theupshotisthatrelyingontoday'scotssystemstoensuresecurityis ariskyproposition,especiallywhensuchsystemsare meanttoworkovertheinternet.thisshortpaper touchesontherisksinherentsomeoftoday'smore popularcotssystems,includingoperatingsystems andjavavirtualmachines.ialsopresentrobustness resultsgatheredbyaresearchprototypecalledrid- DLE(RandomandIntelligentDataDesignLibrary Environment),whichwasusedtoassesstherobustnessofnativeWindowsNTsystemutilitiesaswellas Win32portsoftheGNUutilities. 1COTSinAction (or,cotsinaction) LiketherestoftheDepartmentofDefense,the UnitedStatesNavyismandatedtouseCommercial O-The-Shelf(COTS)technologyinordertostandardizeandtosavemoney.TheNavy'sSmartShip initiative,whichiscurrentlybeingtestedasapilot studyontheaegismissilecruiserussyorktown,is aprimeexampleofthemovetocots.amajorpart ThisworkissponsoredundertheDefenseAdvancedResearchProjectsAgency(DARPA)ContractF C theviewsandconclusionscontainedinthisdocument arethoseoftheauthorsandshouldnotbeinterpreted asrepresentingtheofficialpolicies,eitherexpressed orimplied,ofthedefenseadvancedresearchprojects agencyortheu.s.government. oftheinitiativeistomigratesystemstothemicrosoft WindowsNTOperatingSystem.WhatrecentlyhappenedtotheYorktownservestounderscorethenature ofsecurityrisksinherentincots-basedsystems. InSeptember1997,theYorktownwasunderwayin maneuversothevirginiacoast.duringthemaneuvers,theyorktownsueredaserioussystemsfailure causedbyadividebyzeroerrorinanntapplication. AccordingtotheRISKSdigest(Volume18,Issue88), thezeroseemstohavebeenanerroneousdataitem enteredbyasystemuser.asaresultoftheerror,the shipwasdeadinthewaterforovertwoandahalf hours. Thissomewhatamusinganecdotewouldturnoutto beaveryseriousandpotentiallydeadlyproblemduringwartime.windowsntisknowntohaveanumber offailuremodes,anyoneofwhichcouldbeleveraged intoaninformationwarfareweapon.nevertheless, sincentisquicklybecomingadefactostandardin industry,thedodisunlikelytoabandonitseortto adoptit.insteadofbecominglesslikely,problems suchasthoseexperiencedontheyorktownareahint ofthingstocome. 2COTSProblemsPercolateUp DespitetheproliferationofNTWorkstationsin business-criticalandmission-criticalenvironments,littleanalysisofthesoftwarethatcomprisesthentplatformhasbeenperformed.thisimpliesthattheextent towhichnthasinherentsecurityrisks,systemsbuilt withacotsarchitecturethatincludentinheritthe samerisks. OperatingSystemsarenotaloneinthisproblem. Anythird-partysoftwareincludedinasystemhas thesamerisk-percolationproperty,whetherthesoftwareispackagedatthecomponentlevelorhigher. ThatmeansthatCOTSpartsofelectroniccommerce systemsnowonthedrawingboard,includingweb

2 browsersandjavavirtualmachines,introducesimilarconcerns[2]. IfCOTSintroducemorerisks,whyusethem? Unfortunately,theissueisnotcompletelycut-anddry.COTShaveanumberofimportantbenetsthat shouldnotbeoverlooked.considercomponentre-use inprogrammingsystemslikevisualbasic.today's applicationsaremorecomplicatedthanever,andthe timepressuretogetthemdoneandputthemintouse isgreaterthanever.visualbasiccomponentssave bothtimeandeort.thereisnoreasonthatthe softwareindustryshouldnotlearnfromelectricalengineeringwhereprefabricatedcomponentshavebeen usedforyears. Therealproblemisthis: COTSoftensuerfromdependability,security,andsafetyproblems.Whatcanwedoto analyzecotsandmeasurethemaccording totheseproperties? ThisproblemisexacerbatedbythefactthatCOTS areusuallydeliveredwithnoguaranteesabouttheir behaviorinblackboxform.itishardenoughtotry todeterminewhataprogramwilldogivenitssource code.withoutthesourcecode,theproblembecomes muchharder. 3RisksinJava:Acasestudy TheJavaprogramminglanguagefromSunMicrosystemsmakesaninterestingcasestudyfroma COTSsecurityperspective.Javawas,afterall,designedwithsecurityinmind.Java'ssecuritymechanismsarebuiltonafoundationoftypesafety andincludeanumberoflanguage-basedenforcement mechanisms[4].unfortunately,aswithanycomplex system,javahashaditsproblemswithsecurity.it turnsouttobeveryhardtodothingsexactlyright, andexactlyrightiswhatisdemandedbysecurity. EdFeltenandIhavedenedfourbroadcategories ofattackswithwhichtounderstandjava'ssecurity risks.thesecategoriescanbeusedtocategorizeall mobilecoderisks: 1.Systemmodicationattacksoccurwhenan attackerexploitsasecurityholetotakeoverand modifythetargetsystem.theseattacksarevery seriousandcanbeusedforanynumberofnefariousends,including:installingavirus,installing atrapdoor,installingalisteningpost,reading privatedata,andsoon. 2.Invasionofprivacyattackshappenwhena pieceofjavacodegetsaccesstodatameanttobe keptprivate.suchdataincludespasswordles, personaldirectorynames,andthelike. 3.Denialofserviceattacksmakeitimpossible touseamachineforlegitimateactivities.these kindsofattacksarealmosttrivialintoday'ssystems(javaorotherwise)andareanessentialrisk categoryfore-commerceanddefense. 4.Antagonismattacksaremeanttoharassor annoyalegitimateuser.theseattacksmight includedisplayingobscenepicturesorplaying soundlesforever. Unfortunately,allfourcategoriesofattackcanbe carriedoutinjavasystems.byfarthemostdangerousattacks,systemmodication,leverageholesinthe JavaVirtualMachinetowork.ThoughJava'sinternal defensesagainstsuchattacksarestrong,atleastfteenmajorsecurityholeshavebeendiscoveredinjava (andsincepatched).thelatestsuchhole,aproblem withclassloadinginthejdk1.2beta3,wasdiscoveredinjuly1998. Ifsupposedly-securesystemslikeJavaVirtualMachines(itemscommonlyincludedasCOTSinsystemsrangingfromsmartcardsandembeddeddevices towebbrowsers)havesecurityrisks,whatdoesthis sayaboutlesssecurity-consciouscots?thesomewhatdisturbingansweristhatothersystemsaremuch worseo.microsoft'sactivexsystem,forexample, presentsanumberoffarmoreserioussecurityproblemsthanjavadoes. 4BlackBoxAnalysis Mostsoftwaresecurityvulnerabilitiesresultfrom twofactors:programbugsandmaliciousmisuse. Technologiesandmethodologiesforanalyzingsoftware inordertodiscoverthesevulnerabilities(andpotentialavenuesforexploitation)areacurrenttopicof computersecurityresearch.dynamicsoftwareanalysistechnologiesusuallyrequireprogramsourcecode. However,mostCOTSsoftwareapplicationsaredeliveredintheformofbinaryexecutables(including hookstodynamiclibraries),renderingsource-code{ basedtechniquesuseless.thus,alternativemethods foranalyzingsoftwarevulnerabilityundermalicious misuseorattackarerequired. Dynamicblack-boxanalysisisanimportantapproachtosoftwarevulnerabilitylocalizationthat, giventoday'sinexpensivehardware,canbeperformed relativelycheaply.thisanalysisisavariantontraditionalsoftwaretestingthatisparticularlyattractive becauseitcanbeappliedtobinaryexecutables,includingcotsandlegacyexecutables.thisapproach

3 isnottypicalvanillatesting,butratherfocusedtestingwiththeexpresspurposeofdeterminingacomponent'stolerancetoattack.thoughthisapproach neitherrequiresfunctionalspecicationsforcomponentsnorfunctionalrequirements,itdoesrequirethe usertocharacterizewhatasecurityviolationis(based onsite-specicsecuritypolicy). 4.1RIDDLE:NTrobustness ResearchatReliableSoftwareTechnologiesisaddressingtheproblemofCOTSsecurityanalysisby beginningwiththeproblemofcomponentrobustness [1].Thisfollowsthefootstepsoftworesearcheorts: Fuzz[5]andBallista[3],bothofwhichconsideredthe robustnessofunixsystemsoftware.riddle'starget isnt. TheRandomandIntelligentDataDesignLibrary Environment(RIDDLE)enablesanalysisofcommercialo-the-shelf(COTS)softwarebyusingblack-box testingtechniques.riddlepermitsstresstestingof applicationsoftware,systemutilities,com/dcom components,sharedlibraries,andsystemfunctions. Unliketraditionalblack-boxtestingapproaches,RID- DLEstresstestssoftwareusingunexpected,intelligentlycraftedtestcases.Thegoalofthisresearch istodeterminewhatrobustnessgaps,ifany,existin WindowsNTsoftware. Testcasesaregeneratedwithrandom,intelligent inputusingtheinputgrammarofthecomponentunderanalysis.ratherthansimplygeneratingrandom inputthatdoesnotmeetthebasicsyntaxoftheprogram'sinput,generatinginputintelligentlyusingthe inputgrammarofthecomponentpermitsstresstestingofmoreofthesoftware'sfunctions.riddleprovidesanenvironmenttocombinerandominput,maliciousinput,andboundaryvalueconditionsinthelegalgrammaroftheprogramtotestitsbehaviormore thoroughlyunderanomalousconditions. 4.2Results RIDDLEwasusedtoperformrobustnesstestson twocategoriesofwindowsntsoftware.therst categoryismadeupofwindowsntcommandline utilitiesthataresuppliedwiththeoperatingsystem. Theutilitiestestedareattrib,chkdsk,comp,expand, fc,find,help,label,andreplace.thesecondcategoryofsoftwarethatwastestedwasagroupofgnu commandlineutilitiesthathavebeenportedtothe WindowsNToperatingsystemaspartoftheCygnus project.theportedgnuutilitiestested arecat,chmod,chksum,cp,ls,mv,rm,andwc. Theexperimentationcoveredmanycombinationsof thestringlengthsandcharactersets.inall,therewere 64,000testsrunontheGNUutilities,and114,000 Utility Group Figure1:Percentageofunhandledexceptionsforall testcasesrunagainstthenativewindowsntand utilities.thevastmajorityofunhandled exceptionswerememoryaccessviolationsthatresult intheabortedexecutionoftheprogrambeingtested. testsrunonthenativewindowsntutilities.rid- DLEdetecteddistinctterminationstatesfromthe programsthatweretested.theexitstatesdetermine whentheprogramterminatesnormally,whentheprogramishung,andwhentheprogramterminatesdue toanunhandledexception.threetypesofexceptions werecaughtbytheriddlemonitorintheseexperiments:memoryaccessviolationexceptions,privileged instructionexceptions,andillegalinstructionexceptions.iftheseexceptionsariseduringtheexecution ofaprogram,thentheprogramhasfailedtoperform robustlybyfailingtohandletheexceptioninternally. Figure1summarizestheresultsofthetestingofnativeWindowsNTutilitiesandtheutilities.InallthetestcasesrunagainstthenativeWindowsNTutilities,only0.338%ofthetestcasesresultedinfailureaccordingtoourfailuremetric.On theotherhand,theutilitiesexitwithan unhandledexception10.64%ofthetime.thedistributionofexceptionsfavoredmemoryaccessviolations soheavily(approximately7000to1for, and100to1forwindowsnt)thattheothertypesof exceptionsarestatisticallyinsignicant. Furtheranalysisoftheresultsshow thatthe10.64%failurerateisfairlyconsistentacross theeightgnuutilitiesthatweretested.thevast majorityoftheexceptionsoccurredwhenthechar-

4 Figure2:Distributionofunhandledexceptionsamong Figure3:Thepercentageoftestcasesthatresulted 2.0 Alphabetical Printable character. rangeofthelastcolumn,[0,255],includesthenull cludesallcharactersexceptthenullcharacter.the dierentcontenttypes.thecharacterrange[1,255]in- Character Set dramaticallywhenthecharactersetisalteredtoincludethenullcharacter,orwhenitconsistsonlyocialcharacters.thenumberofexceptionsdecreases mostlikelyduetotheprogram'sinterpretationofspe- range[1,255](excludingthenullcharacter).thisis actersetbeingusedforstringgenerationwasinthe input.thiswouldexplainwhytherearefewerunhandledexceptionswhenthischaracterisusedinlighacterofastring,eectivelylimitingthelengthofthe charactermaybeinterpretedastheterminationchar- printablecharactersintherange[33,127].thenull Figure3).Anotherpossibilityisthatifthenullcharacterisinterpretedaseithertheendofastringorthe ofthecorrelationbetweenlengthandexceptions(see endoftheparameterlist,thentheparametersmay theutilitymayimmediatelyrejectthetestcase. nolongerconstituteavaliduseoftheapplicationand eryprintableandnon-printablecharacterexceptfor thecharactersetrange[1,255].thissetincludesevtiesaremostvulnerabletoinputthatissampledfrom showninfigure2.clearly,theutili- Thedistributionofexceptionsbycontenttypeis set(includingnon-printablecharacters)thatresulted lengthwithnearlytheentirerangeofthecharacter ceptions.instead,itisthecombinationofverylong thealphabeticalandprintablesetresultedinfewex- thenullcharacter.evenverylonglengthinputin 1.0 inexceptionsasafunctionofthelengthoftheinput strings String Length (Characters) inthemostunhandledexceptions. oftheexceptionratiosshowthatasthe stringincreasesasillustratedinfigure3.thegraph istheincreaseinunhandledexceptionsasthelengthof fromthetestsperformedontheutilities Themostsignicanttrendinthedatacollected whenthelengthofthestringusedwaseither8or putgrammar.signicantlyfewerexceptionsoccurred failuretohandleanomalousinputwithinproperin- lengthofinputisincreasedfrom8to4096bytes,the 250characters.Becausetheexceptionthatoccurred numberofexceptionsrisesdramaticallyindicatinga longinputprobablypointstoaregionofthememory theinstructionpointerthatwasoverwrittenwiththe legalpointerontheprogramstack.inotherwords, ismostlikelyanover-writtenbuerthatplacedanil- mostoftenwasamemoryaccessviolation,thecause thatisinaccessiblefortheprogram,oritmaypoint todatathatisnotavalidinstructionopcode.this utilitiestobueroverrunattacks. resultpointstopotentialvulnerabilitiesinthegnu Theexpandutilityhadafailurepatternsimilartothe ofthem,compandexpandproducedanyexceptions. ture.ofthenineutilitiesthatweretested,onlytwo tivewindowsntutilitiespaintsaverydierentpic- Thedatacollectedfromthetestsrunonthena- complex.thecomputilityfailedmostfrequentlywhen stringswerelongerandthecharactersetsweremore utilities.itfailedmoreoftenwhenthe

5 was250. 5TowardsManagingCOTSRisks thecharactersetwasalphabetic,andthestringlength inmind(likethejavavm)suerfromserioussecurityproblems,wecanonlycringeatthethoughtof GiventhatCOTSspecicallydesignedwithsecurity COTSarebecomingasubiquitousassoftwareitself. derstandingthesecurityimplicationsofusingcots. Itisclearthatmuchworkremainstobedoneinun- therisksthatlesscarefully-designedcotsintroduce. desktopapplications,andole/com/dcomcomponents.futureresearchwillinvolvetestingtheseother supporttestingofnetworkservers,sharedlibraries, behaviorofablackboxcotssystemisauseful exercise.riddleiscurrentlybeingexpandedto TheRIDDLEexperimentsshowthatprobingthe securityholes. gapstodeterminetheirpotentialtobeexploitedinto classesofntsoftwareaswellasexploringrobustness AcknowledgementsTheRIDDLEworksketchedherewas ReliableSoftwareTechnologies.See[1]foramorethorough performedbyanupghosh,mattschmid,andvirenshahof [1]A.Ghosh,M.Schmid,andV.Shah.Testingthe References treatmentofthesubject. [2]A.K.Ghosh.E-CommerceSecurity:WeakLinks, November robustnessofwindowsntsoftware.toappear, [3]P.Koopman,J.Sung,C.Dingman,D.Siewiorek, BestDefenses.JohnWiley&Sons,NewYork, robustnessbenchmarks.inproceedingsofthe16th andt.marz.comparingoperatingsystemsusing NY,1998.ISBN [4]G.McGrawandE.Felten.JavaSecurity:Hostile pages72{79,october1997. IEEESymposiumonReliableDistributedSystems, [5]B.P.Miller,D.Koski,C.P.Lee,V.Maganty, R.Murthy,A.Natarajan,andJ.Steidl.Fuzzrevisted:Are-examinationofthereliabilityofunix ofwisconsin,computersciencesdept,november Sons,NewYork,1996. Applets,Holes,andAntidotes.JohnWileyand utilitiesandservices.technicalreport,university