Adelaide,December7,2001 KendyKutzner

Telecommunications Institutefor Research forfiredetectioninaustralia ProcessingMODISData Prof.WilliamCowley,ITR KendyKutzner MichaelSloman,ITR Supervisors: InformationEngineering SchoolofElectricaland December7,2001

publishedorwrittenbyanotherauthorexceptwhereduereferenceismade inthetext. Ideclarethatthisworkdoesnotincorporateanymaterialpreviously2 Adelaide,December7,2001 KendyKutzner Australia,inordertofullltherequirementsfor Studienarbeittoreachthedegreeof'DiplomingenieurElektrotechnik' ThisworkissubmittedtotheInstituteforTelecommunicationsResearch, Thehonorspartoftheworkisespeciallypresentinsection4. Finalyearproject(hons)attheschoolofElectricalandInformation atthechemnitzuniversityoftechnology,germany Engineering,UniversityofSouthAustralia

ListofFigures TableofContents CONTENTS 43 2Introduction ListofTables 1Abstract 2.3TerraandMODIS...7 5 2.4ConventionsandAbbreviations...8 2.2ITRandASTRA...7 2.1SouthAustraliaandBushFires...776 2.5OverviewoverDataow...8 2.6UsedFileFormats...9 2.6.1Rawrecordeddata...9 3DataProcessingtoLevel1b 2.7BowTieEectandMapProjections...11 2.6.2ProductionDataSets...9 2.6.3HierarchicalDataFormat...10 3.2IntroductionintoFrameSynchronization...13 3.1Preprocessing...13 2.6.5FileSizes...11 2.6.4OtherusedFileFormats...10 3.3TheProgramFrameSync...16 3.4FrameSynchronizationwithSTPS...16 3.2.1BitSynchronization...13 3.2.2FrameFormat...14 4FireDetection 3.4.3stpsccanddltp...20 3.4.1reformat...17 4.1AlgorithmsforPreparingFireDetection...23 3.5IMAPP...22 3.4.2f2s...19 4.1.2CreationandEvaluationoftheSnowMask...24 3.4.4pk...22 4.1.3CloudDetection...25 4.1.1EvaluationoftheLand/SeaMask...24

CONTENTS 4.2FireDetectionAlgorithms...25 4.3AlgorithmsusingtheResultsofFireDetection...27 4.4ImplementationofFireDetection...29 4.3.1MapProjection...27 4.3.2Smoothing...284 6Recommendation 5Conclusions 34 CMatlabSourceFilesforFireDetection BGlossary ADimple C.1firedetection.m...40 39 38 37 C.2radiance2teff.m...47 EOthersourcecodes DAdditionalMatlabsourceles D.2hdf2bin.m...52 D.1all500m2tiff.m...51 C.4sdsread.m...49 C.3attributeread.m...48 E.1FrameSync.c...57 E.2slicer.c...60 51 FCongurationFiles References E.3Theshellscript...62 F.2stps.cfg...64 F.3pk.cfg...66 F.1socapps.cfg...64 68

LISTOFFIGURES ListofFigures 1FlowofdatafromASTRAtoredetectionresult...9 2BowTieEect...12 3DownlinkPacketFormat...155 ListofTables 4DataowinSTPS...18 5Sampleoutputfromthescriptfiredetection...31 2Someimportantsettingsinthecongurationleofstpscc...21 1ExamplesofFileSizes...11 6Pictureofdetectedres...35 3Possiblevaluesforland/seamask...24 7Listofdetectedres...36

mentoftheterrasatellitetodetectbushresinaustralia.thisincluded preprocessingthedemodulatoroutput,bitsynchronizationandreassembly 1ABSTRACT 1Abstract TheaimofthisworkwastouseremotesensingdatafromtheMODISinstru- 6 ofdatapackets.imappwasusedtodothegeolocationanddatacalibration. similar. ferencetestsandbackgroundcomparisons.theresultswereprojectedina bynasaandredetectionsbasedonothersensorsandfoundtobevery rectangularlatitude/longitudemaptoremedythebowtieeect.algorithms intheavailabledata.theresultswerecomparedwithredetectionsdone wereimplementedincandmatlab.itprovedtobepossibletodetectres Theredetectionusedacombinationofxedthresholdtechniqueswithdif-

Itbeginswithageneralintroductionwhyremotesensingisusefulforredetection.Thentheusedconventionsandleformatsareexplainedandatthe 2INTRODUCTION 7 2Introduction Thissectionshouldgiveanoverviewoverproblemsthisworkisdealingwith. continent.itslargeagricultural,shrubandforestedareasposingahighre SouthAustraliaisthethirdlargeststateinAustraliawithanareaofmore 2.1SouthAustraliaandBushFires endsomeoftheproblemsaredescribedindetail. dicultbecauseofthevastareaandsparsepopulationinmostofthestate. than9:841011m2[7].itisoftenreferredasthedrieststateinthedriest supporttheearlyredetection. hazardinmostsummers.therewere3204hectaresburntandover5million 2.2ITRandASTRA AustralianDollarslostinruralresinthe1999/2000summeralone[3]. Thisworktriestodeterminethepossibilityofusingremotesensingdatato Thisshouldmakecleartheneedforfastredetection.Firereportingis TheInstituteforCommunicationsResearch(ITR)hasgottheASTRAsince ableparabolicdishandiscurrentlyconguredforxband.itispossibleto December1984[11].ASTRAstandsforAutomaticSatelliteTrackingResearchAntennaandisoriginallyfromNASA.Itisa6.8mhydraulicalsteer- SATandRADARSAT[11].Inthisworktheinterestliesonthesatellites receiveearthresourcesatellitessuchasers,jers,spot,mos,land- Terra(formerlyEOS-AM)andAqua(EOS-PM). TheTerrasatelliteispartofNASA'sEarthObservingSystem(EOS).Itwas 2.3TerraandMODIS 150megabitperseconddevelopedatITR. withersdem2,acongurabledemodulatorcapableofdatastreamsupto AfterreceptionwithASTRAthesignalgetsimmediatelydemodulated launchedindecember1999andisreferredastheagshipoftheeos.since themoderateresolutionimagingspectroradiometerormodis.thissensor seestheentireearthsurfaceeverytwodaysandacquiresdatain36dierent February2000itisfullyoperational.Onofthemaininstrumentitcarriesis bands.

alreadygoodcoveragewillbeevenbetter.untilnow,everydayoneoverpass ment.aquawillbelaunched,accordingto[23],inmarch2002.thenthe phenomenology.theeos-pmoraquaplatformwillcarrythesameinstru- 2INTRODUCTION ofitschannelsareespeciallydesignedforterrestrial,atmosphericorocean MODISismeanttobeofuseforaverybroadrangeofscientists.Many8 castfacilityorcanbestoredandtransferedtotheearthlaterindirectplay- backmode.itrhasthecapabilitytoreceivethecontinuousdirectbroadcast datastreamfromthemodisinstrument. canberecordedinthemorning,withaquatherewillbeoneintheafternoon (That'swhythesatellitewascalled-PM). DatacollectedbyMODISistransmittedimmediatlyviathedirectbroad- ForalistofabbreviationspleaserefertoappendixB. 2.4ConventionsandAbbreviations maybeprovideusefulresultstoorganizationslikethecountryfireservice. Thegoalwastoautomaticallydetectbushresfromreceiveddata. Thisworkisthetrialtoprocessthedataabovethelevel1bstageand puterunlessotherwisestated. Themostoftenusedsymbolsinclude: 'CommandLine'usuallyreferstoashshellonaSunOSorLinuxcom- BRadianceatwavelength TBrightnesstemperatureconvertedfromB Reectanceatwavelength Centerwavelengthofachannel(inm) Latitudevalues dentofeachotherdierentlevelshavebeendenedandwidelyaccepted. 2.5OverviewoverDataow havetobeprocessedafewsteps.tomaketheseprocessingstagesindepen- Firedetectiondoesnotworkimmediatelywiththereceiveddata.Theles Longitudevalues Level0dataisdenedasthereconstructedCCSDSpackets(seesection3.2.2) 2.6.3)producedbyIMAPP(seesection3.5).Level1bdataistheradiometricallycalibrateddata.Furthermoregeolocationinformationisavailablein removed.level1aisanintermediateproductbasedonhdf(seesection thelevel1bproduct.thisincludesancillarydatalikethepositionofthe Rawdataisjustrecordedfromthesatelliteandnotprocessedinanyway. withallcommunicationartifactsincludingduplicateanderroneouspackets spacecraftorsensorandsolarangles.

Thisworkstartsatthepreprocessingstageswhenrawlesaretouchedfor anddemodulatedwithersdem2.thesestepsarenotpartofthisreport. detection. 2INTRODUCTION Inlevel1bproductsenoughinformationisavailabletodoasuccessfulre Theowofdatacanbeseeningure1.RawdataisreceivedbyASTRA9 clusionofobviouslybaddataandthebitreversion.level0dataisproduced thersttimeintheconversiontolevel0data.preprocessingincludestheex- byframesyncorstps(seesections3.3and3.4).imappisresponsiblefor stepstolevel1b.theactualredetectionisbasedonthisoutputandis doneinmatlab. Figure1:FlowofdatafromASTRAtoredetectionresult 2.6UsedFileFormats 1 0 slicer & reformat 2.6.1Rawrecordeddata raw AfterreceptionofthedatabyASTRAanddemodulationbyERSDEM2 FrameSync STPS ASTRA dataisrecordedtocomputerles.theseleshavenoparticularformatand level 0 IMAPP level 1 broadcastmodeandaaverageoverpassdurationlike12minutesthetypical DuetotheTerradatarateof13.125megabitpersecondintheuseddirect containaconcatenationofbits.usuallytheselescarrytheextensionraw. Dimple detected fires lesizeisaroundonegigabyte.amoredetaileddescriptionofthedataand Matlab (picture and list) consecutiveccsdspackets(seesection3.2.2).thesepacketstreamshould problemsassociatedwithitareinsection3.1. 2.6.2ProductionDataSets Datainlevel0isoftencalledproductiondatasets.Thesedatasetsconsistsof

notcontainanyartifactsfromthecommunicationanymore.allerroneous anddoubledpacketsshouldberemoved. 2INTRODUCTION ditionalheaderscontainingmetainformation(edosheaders),butinthis projectimapp(seesection3.5)isusedforlevel1processinganddoesn't SomeprogramslikefurtherprocessingsoftwarefromNASArequiresad- 10 2.6.3HierarchicalDataFormat.ccsdsbecausetheycontainCCSDSpackets. smallerthanrawdatales(seesection2.6.5). needsuchheaders. Theextensionoftheselesisusually.pdsforproductiondatasetor Sincetheselesdon'tcontaintheprotocoloverheadtheyarealittlebit umentedandsupportedandmainlyusedforscienticdataexchangeand forsupercomputingapplications.itisaopenleformat,itiswelldoc- archival. TheHierarchicalDataFormat(HDF)wasdevelopedbytheNationalCenter oneishdfversionve.inthisprojecthoweveronlyversionfourisused, format. itisstillfullysupportedbyncsaandmanyavailabletoolsoperateonthis Therearetwoentirelydierentandincompatibleformats.Thenewer bandsarestoredtogetherinonedataset. sets.herethemaximumdimensionusedisthree,sincemanytwodimensional scriptionofdatasetsorstorageofadditionalscalingvalues. attributeswiththedata.thisisusedforexampleforhumanreadablede- OneofthemainfeaturesofHDFisitssupporttostoremetadataor theprovidedinterface.allaccessestothedataaredonethroughalibrary andnolowlevelknowledgeoftheleformatisnecessary. AnadvantageofHDFoverotherleformatsisthestrictneedtoused Anotherinterestingfeatureistheabilitytostoremultidimensionaldata 2.6.4OtherusedFileFormats Partsofthisprojectrequirethestoreofimagedata.Forthispurposethe intheunixwayforlineendingswith0x10asalineendmarker.toconvert tionlesmentionedandallsourcelesformatlabandc.textlesarestored andstores24bitcolorimages. formattiff(taggedimagefileformat)waschosen.thishasseveral platformsandsoftwarepackages.itiscapableoflosslessdatacompression reasonsincludingthewidesupportforthisformatacrossdierentcomputer Textlesincludetheoutputoftheredetectionalgorithm,thecongura-

resonecantype thatfurtherprocessingiseasy.forexampletogetthenumberofdetected themtoanydos/windowsbasedplatformthetoolunix2dos<filename> canbeused. 2INTRODUCTION Thetextleproducedbytheredetectionscriptareformatedinaway 11 wellasthepositionoftherepixelinthenalmap. grep-v^#lenamejwc-l 2.6.5FileSizes lineisprinted.itcontainsthelatitudeandlongitudevaluesofthepixelas outputlenamesarestated.aftertheheaderforeachrepixelfoundone atthecommandlineprompt.theleheadercontainsinformationabout theprocessingdoneandtheproportionsoftheproducedimage.inputand withtheselesmustbeawareofthis. getanideaoflesizes,seetable1.allprogramsandalgorithmsdealing Remotesensingdatatendstobelarge,especiallywithhigherresolution.To recordingdate29/11/200105/12/2001 afterframesyncb330,155,562250,984,764 level1a geolocationdata50,765,78338,511,453 1000mdata initiallesizea411,041,792312,475,648 287,492,136218,165,563 98,713,60477,012,996 Table1:ExamplesofFileSizes 500mdata 250mdata resulting.tile8,072,2863,074,088 arawdata blevel0 230,296,084174,759,533 239,500,060181,742,949 Tieeect.Itwasrecordedon30/08/2001andshowstheEerieandYork andtoadistortedimageattheedges.figure2isanexampleofthebow TheMODISinstrumenthasthenominalspatialresolutiononlynearthe thandoubleofthenominalone[26].thisleadstooverlappingofswaths nadirline.asaresultofthemotionofthescanningmirrorintheinstrument, dataelementsattheedgeofthe110degreeswathwidthhaveasizeofmore 2.7BowTieEectandMapProjections

PeninsulainSouthAustraliainthe500muncalibratedcolor.Notethewrong aspectratioasaresultofnonsquarepictureelementsfrommodis.another unpleasantartifactisthattheimageorientationisnotnorthbutintheight pathofthesatellite. 2INTRODUCTION IMAPPprovidesthegeolocationdataforevery1kmpixel.ThisinformationcanbeusedtocorrectbothBowTieanddistortioneects. TransverseMercator)andrectangularlatitude/longitude.Goodexplanation dierentapproachesareknown.twocommononesaretheutm(universal Rectangularlatitude/longitudemapsthesphericalsurfacetoarectangle Earth'snearsphericalsurfaceisnoteasilyprojectedtoaplane.Many 12 canbefoundin[27].forconversionseeforexample[29].therstprovides muchbetterresultsforsmallerareasbutprovednotpracticalforrecorded imageswithsizesupto3420x2332km. disadvantageofdistortionnearthepoles(apointismappedtoaline)but withlatitudeandlongitudeasxandycoordinates.thishastheapparent thisisnotahugeprobleminthelandareaofaustraliawithlatitudevalues lessthan40degrees.figure2:bowtieeect

3DATAPROCESSINGTOLEVEL1B 3.1Preprocessing Afterrecordingtherawdatafromthesatellitesdirectbroadcastfacilitya 3DataProcessingtoLevel1b 13 numberofprocessingstepsarenecessary.therstisthepreprocessingto getridoflargeramountsofbaddatarecordedfromthedemodulator.there Itperformsakindofanautocorrelationoftherawdata.Ifthiscorrelationpeaksat1024becauseoftherepeatedsyncmarkers(section3.2.2),the programassumesgooddataandpassesthedatatothenextsteps. Thisisdonewiththesmallprogramslicer,whichisbasedonasimilar forsomereasonorwhenthesatellite'stransmitterisswitchedoforthe programfromrickyluppino.sourcecodecanbeseeninappendixe.2. receptionofdeepspaceprobes. Furtherbaddatablocksmaybetherewhenthetrackingofthesatellitefails areblocksofunusabledataatthebeginningandtheendofeachrecording. availableinitr.theprogramreformat(seesection3.4.1)cancelsthisreversion. 3.2IntroductionintoFrameSynchronization scribedinsection3.4.thisprogramcan'tdealwithbytereverseddata AnotherpreprocessingisnecessaryfortheuseoftheSTPSpackagede- issaved.anothernameforthistypeofdataisrawtelemetrychanneldata. Asdescribedbefore,theoutputofpreviousstagesistheconcatenationof thelestherawrecordingfromallbroadcastcommunicationofthesatellite havetobeunderstoodandthedataweareinterestedincanbeextracted. bits.untilnow,noalignmentisdoneandnoneofthebitshasameaning.in 3.2.1BitSynchronization Units(CADU)orframes.AllCADUshavethesamelengthof1024octets. DatainthesatellitedownlinkisorganizedinsocalledChannelAccessData Thenexttaskistobytealignthesebits.Afterthis,theprotocolsused Tosupportthesearchforframestarts,therstpartofeachframeisan AttachedSyncMarker(ASM)of32bit.Thechosenbitpatternforthe channelsbyconsultativecommitteeforspacedatasystems(ccsds).it Terrasatelliteis00011010110011111111110000011101orinhexadecimal notation0x1acffc1d.thissyncmarkerisrecommendedfortelemetrydata willbeusedbyothersatelliteslikeaqua.bitsynchronizationisconsidered

3DATAPROCESSINGTOLEVEL1B donelike donewhenthesyncsequenceisrecognized.however,furthercheckscanbe Arethereothersyncmarkers1024octetsbeforeandafterthisposition? 14 demodulatorcongurationsthepolarityofbitsmaybeinverted.theknown patterncanbeusedtodetectthisconditionandswitchallthebitsback. WithERSDEM2andTerrathisproblemdoesnotoccur. Doesthecontentoftheframemakessensetothenextprotocollayers? Doesthecontentoftheframepassesanyerrortest? 3.2.2FrameFormat Anothertaskcanutilizetheattachedsyncmarkerpattern.Insome ThenextstepistheextractionofMODISpackets,sincethedatastreamuntil nowmaycontaindatafromotheron-boardinstrumentsorothertelemetry moved. data.moreovertheprotocoloverheadforthetransportlayerhastobere- calledchannelaccessdataunit(cadu).eachcaduhastheconstant sizeof1024byte.therst32bitsofeachcaduistheattachedsync marker.thesecondpartisexactlyonecodedvirtualchanneldataunit (CVCDU)of1020bytes. Asmentionedbefore,datafromthespacecraftisorganizedinframes satellitedownlink). and128bytesappendedreedsolomonforwarderrorcorrectioncode.the datachannels(forexampledierentinstruments)overonephysicallink(the purposeofvirtualchannelsisthepossibilitytotransmitseveraldierent EveryCVCDUconsistsof892bytesVirtualChannelDataUnit(VCDU) CCSDSpackets. spacecraftidassignedbyccsds.terrahastheid0x2a,aquawillhave 1intheTerraandAquasatellites.Thenexteldstatesthegloballyunique theid0x9a[8]. intothevcduheaderandampduheader.thesecondpartconsistsof TheVCDUheadercontainstheversionnumberoftheprotocolwhichis Thevirtualchanneldataunithastwomainparts.Therstpartdivided thesamechannelnumberformonevirtualchannel.fortheassignmentof foremptyvcdus. channelnumberssee[19]page61.aspecialchannelidis0x3fwhichisused channel.allvirtualchanneldataunitswiththesamespacecraftidand Thespacecraftidenticationeldisfollowedbythenumberofthevirtual

3DATAPROCESSINGTOLEVEL1B inthetelemetrydatastream. channel.withthiselditispossibletodetectduplicatedormissingparts Thenexteldisacountofvirtualchanneldataunitsforeachvirtual 15 Figure3:DownlinkPacketFormat end of previous CADU CADU (1024 octets) start of next CADU Attached Sync Marker (4 octets) Coded VCDU (1020 octets) canhavedierentlengths(forexampleday,nightorllpackets),eachpacket VCDU (892 octets) Reed Solomon Code (128 octets) header (6 octets) data (886 octets) thetaskoftherawtolevel0softwaretoreassemblethesesfragments. CCSDSpacketintheMPDUheader.ItispossibleforaCCSDSpacketto besplitovermultiplevcdus.eventheheadereldofaccsdspacketmay carriesitsownlengtheldandthereisapointertothebeginningoftherst bedivided.itisalsopossibletohavemorethanonepacketpervcdu.itis AftertheMPDUheadertheCCSDSpacketsfollow.Sincethesepackets M_PDU header (2 octets) end last packet(s) (variable size) start next transitiondensityinthedownlinkstream,bitsarexor-edwithapseudo randomnoisesequenceh(x)=x8+x7+x5+x3+x1 Toavoidrepeatingpatternsbesidesthesyncmarkerandensuredata Primary Header Secondary Header MODIS header MODIS data Checksum errorcorrectiontoformacvcdu. [20].ThissequenceisinitializedtoallOne'satthestartofeveryVCDU. InordertosuccessfullydecodeframesthestepsareforTerra ThesequenceofthetwodierentcodingsvariesbetweenTerraandAqua. EachVCDUisfollowedby128octetsReedSolomoncodeforforward

3DATAPROCESSINGTOLEVEL1B whileforaquathestepsare 2.PNDecodeVCDUbutnottheVCDUheader 1.DecodeReedSolomon(RS)forentireVCDU(CADUwithoutsync markerandrscode) 16 3.3TheProgramFrameSync Toperformthetasksdescribedabove,theprogramFrameSyncwaswritten attheuniversityofhawaii.theoriginalversiondidnotworkverygoodon 1.PNdecodeCVCDU(includingVCDUheaderandRScode) thedataavailableatitrsoafewchangesweredone. 2.DecodeRSforentireVCDU. with16bitdata,buttheorginalversioncheckedonlyforeightdierentbit ternappearsatthebeginningofablock.thechangedvariantshiftsthe syncmarkerthroughallpossibilitiesandcomparestheorginaldatawiththe shiftedsyncpatterns,whichismuchfaster.internallytheprogramworks originalversiondidthisbybitshiftingthewholedatauntilthemarkerpat- shifts.obviously16dierentbyteshiftsarepossiblewithwordsizedata. Atrst,theprogramstartstolookforthesyncmarkerpattern.The Thesearchfortheremainingeightcontingencieswasadded. CCSDSpackets.Theseincludechecksforpacketsizeandcorrectnessof subjoined. headerelds.tosupportthedebugingafacilityfordumpingvcduswas Alsoaddedwereanumberofcheckstopreventtheoutputofinvalid 3.4FrameSynchronizationwithSTPS andadierentspacecraftidwillbeused. satellitewillbelaunchedbecausethedecodingsequenceisslightlychanged suchdatawasnotavailable.furtherchangesmustbedonewhentheaqua TheprogramFrameSyncwasnottestedwithMODISnightpacketssince Asdescribedbefore,theprogramFrameSyncasseveraldisadvantages.Althoughtherehavebeensomeimprovements,afewbugsremain.Forexample, (Theformatofthedatastreamisexplainedin[10].)IfaCCSDSpacket syncmarkeratthebeginningofit.theprogramignoresthereedsolomon codesprovidedbythesatellite.anotherwayofcheckingframeswouldbe lookingforthenextsyncmarker,butframesyncdoesn'tdothis. anydamagedframeisstillregardedascompleteaslongthereisaattached Atthenextstage,CCSDSpacketshavetobeassembledfromtheVCDUs.

3DATAPROCESSINGTOLEVEL1B Thenextarrivingfragmentofapacketisappendedtothersthalf,ifthe isnotnishedinthecurrentframe,framesyncwaitsforthesecondhalf. VCDUcounteristhesuccessorofthepreviousone.Thisworksne,because above,thedatausedtocompletepacketsisnotchecked.forexample,ifthe packagesfromallothervirtualchannelsarediscardedearlier.asmentioned 17 packetwouldresult.alengthcheckisdone,butmodisnightpacketswill pointertotherstccsdspacketinthesecondframeiscorrupt,ainvalid (GSFC)provedtobeabletosolvesomeoftheproblems.STPSstandsfor SatelliteTelemetryProcessingSystem. theendofeachmodispacket. failthischeck. inthepackagestpsv3.41.tar.gzfromnasagoddardspaceflightcenter Theoveralltaskisnotdonebyasingleprogram.InNASA'sapproach That'swhyanotherwayofdoingthistaskwasguredout.Thesoftware Finally,thepacketsarewrittenwithoutcheckingtheavailableCRCat shouldbe reformatisnecessary. theprogramsf2s,stpscc,dltpandpkareinvolved.additionallythetool 'exportstpslocation=/home/.../stps'. variableintheenvironmentpointstothebasedirectoryofstps.inthecsh ortcshshelltheneededcommandlookslike 'setenvstpslocation/home/.../stps'.withtheshorbashshellsit Forallaboveprogramsexceptreformatandf2sitisnecessarythata onepage18. 3.4.1reformat Itwasfoundthatthe.rawlesavailableinITRarebyte-reversed.Thisis notaproblemfortheprogramframesyncbecausetheswitch-rismade Theinteractionbetweenthedierentprogramscanbeseeningure4 forthiscondition.howevertheprogramdltpcannotdealwithsuchlesso whichcandothistask.ifthebitsinthelesareordered1-2-...-7-8-9-10-...-15-16-17-...,afterrunningthroughthetoolthesequenceischangedto thehavetobeturnedexternally.michaelslomanwrotethetoolreformat 8-7-...-2-1-16-15-...-10-9-17... reformat<inputle>outputle entirelesbytebybyte,itdoesnotmatterwhethertheexecutingcomputer islittleendianorbigendianmachine. Theprogramactsasalter,soacommonwaytocallitis Sincetheinputlesarenotbyteorwordswappedandreformatreadsthe

3DATAPROCESSINGTOLEVEL1B 18 Figure4:DataowinSTPS bit reversed.raw file f2s on ingest TCP port 8003 stpscc on nugget dltp on nugget TCP port 2005 pk vc42ap64.dat on nugget

3DATAPROCESSINGTOLEVEL1B 3.4.2f2s Thetoolf2sreadsalefromdisk,waitsforaconnectionattemptona speciedtcpportandahitfromthekeyboard.aftertransmittingthele viathisconnection,theprogramexits. 19 dltpandpkcanonlyrunonbigendiancomputers,sothedatahastobe transferedtheresomehow.secondtheprogramdltpdoesn'treadfromdisk les,onlyfromtcpports. ofsuchaleisincludedinsectionf.1.forf2sitisnecessarythatthele containsalinestartingwithf2sandonestartingwith*end*.anylinenot Therearetworeasonsforusingthistool.Firstthenextstageprograms startingwithakeywordisregardedascomment. arenameoftheremotehost,wheredltpisrunning.itcanbeanipaddress. Theprogram'sexecutioniscontrolledbyacongurationle.Anexample TCPportnumberfortheconnection.8003isrecommended. Blocksizefortransmission.Smallvaluesincreasetransmissiontime. AfterthekeywordF2Stherehavetobesevenparameters.Theirmeaning Numberofrepeats.Inthisapplicationitisnotusefultosetittoother Waittimebetweenpacketsinmilliseconds.Thisvaluecanbeusedto valuesthan1. reducesystemloadonthetargetcomputer. more. 32000islargeenoughsolargernumbersdon'tspeeduptheprocessany transmitadditionaledos-headerinformationinthetcpstream.since Modeofoperation.Avalueof0meansopeningthesocketinserver dltpgetsconfusedwiththisinformation,itsuseisnotrecommended.other Ifthelastvalueissetto1insteadof0,f2soutputsafewmorestatus Thereareslightlydierentvariantsoff2s.Thevariantscallededosf2s mode,1clientmode.hereonlyservermodeisused. dierencesarewhetherf2sreadsintheentireleatstartuporreadsitin inblocksasnecessary.theauthoroff2sclaimstherstmethodwouldbe messages,whichcanbeuseful. memory,thatswhythesecondmethodhasbeenused. faster.thiscouldnotbereproduced.thelesarelargecomparedtosystem

3DATAPROCESSINGTOLEVEL1B 3.4.3stpsccanddltp istheircloseinteraction.thechannelcontrollerstpsccisresponsiblefor Thereasonwhythesetwoprogramsaredescribedtogetherinonesection readinginthecongurationle,startingandstoppingthechildprocessdltp 20 andprintingoutprogressinformation. nicatingviaudpwithagraphicaluserinterface.thesefeatureshavenot acongurationle.onthecommandlinetherehavetobefourparameters. beenusedhere. IthassomeothercapabilitieslikereadingfromDSPdevicesorcommu- itaterminatingsignaleitherviacontrol-corviathekillcommand.ifstpscc arenecessaryforasuccessfulstart,atinyshellscriptrunstpsisprovided. nicationwiththeguiandarenotimportanthere.becauseallparameters Therstonenamesthecongurationle,alltheothersdealwithcommu- Asusual,stpsccisconguredviaparametersonthecommandlineandvia receivessuchasignal,itshutsallopenconnectionsdownandkillsdltp.then theprogramexits. Oncestarted,stpsccrunsforever.Thewaytoendtheprogramistosend stepisthereedsolomondecoding.anumberofbiterrorscanbecorrected. modules,wherepnderandomizationisdone.intheoutputmodulethenext Iftheremoreerrorsthancanberecovered,theframeisdiscarded. thelockstateisentered.thenframesareforwardedtotheedosoutput fromf2s.afterthesyncpatternisfoundseveraltimesattheexpectedplace AftertheframewassuccessfulPNandRSdecoded,itsendviaanother Theimportantprogramhereisdltp.ItreceivesviaaTCPlinkthedata TCPlinktopk.InthisdatastreaminfrontofeachframeaEDOSheader workinglealllineshavetobeincluded,eveniftheyhavenomeaningin issend,whichisnecessaryforpktounderstandthem. thiscase,likethelinesaboutthedspprocess. thelearenotallowed. lineshavetoappearandthesequencemustnotbechanged.commentsin F.2.Thewaystpsccreadsinthisleisnotespeciallysmart,thatswhyall Intable2theexplanationofsomeimportantlinescanbeseen.Ina Anexampleofaworkingcongurationlehasbeenincludedinsection OnLinuxcomputersitcanbecompiledonlywithchangingtheMakelesand somesourcecodelines.itdoesn'trunwellonlinuxmachines. ThissoftwarewasdevelopedonSunmachinesusingtheGNUCcompiler.

3DATAPROCESSINGTOLEVEL1B 21 errorlogledir.. VERSION3.41 Table2:Someimportantsettingsinthecongurationleofstpscc Necessarybecausetheformatofthiscongurationlechangedbetweenversions inputdatatype0 inputbuersize statuslogledir.. dspprocessup0 Indicatesdataisnotformatedinanyway.If Herenoattemptismadetoreaddirectlyfrom Adirectorywheresomeprogressinformation Amountofdatareadinatonce DSPs abletoreaddata. thissettingisnotcorrect,dltpwouldnotbe arestored. dltpframedatatole0becausedataissendtothenextstageitisnot checkinputblockcrc0framesarenotprotectedbyacrc. dltpup1 Thislinebringsupthedltpprocess. fsoutputframesstate2meaningofstateormode:0-search,1-check, fspndecodeccsds0 dltpsockethostmachinewheref2sisrunning fsbitreverseframes0 PNdecodingisdonelaterintheEDOSmodule,whichisconguredattheendofthele. necessarytosaveit. notthesameasreformat EdosDiscardBadFrames1 checkframecrc0 rsdheaderonly0 EdosOutputServerHostThecomputerpkisrunningon rsdenabled1 fsframespermode19 Framesdon'thaveCRC's 2-Lock,3-Flywheela. EdosOutputSync1 TerraSpecicPNdecode1 aflywheelmodeindltpisnotcomparabletoywheelmodeinframesync

3DATAPROCESSINGTOLEVEL1B 3.4.4pk ThepacketprocessorpkisusedtoextractCCSDSdatapacketsfromthe streamofframes,reassemblethemandseparatethemfordierentvirtual channelandapplicationid.herevirtualchannel42andapplication64 22 lesarenamedvc<vcid>ap<appid>.datsoheretheoutputlecalled thecongurationleisgiveninsectionf.3. file>set.ifnolenameisgiven,thedefaultispk.config.anexampleof startedfromthecommandline,usuallywiththeswitch-s<configuration (MODIS)inthischannelareofinterest. vc42ap64.datistheinterestingone. ForeachchannelandeachapplicationIDaseparateleiswritten.All TheprogramresidesinthesubdirectorypkSERVICEPROCESSOR.It thenextstageofdataprocessinganddoesn'tunderstandthem.that'swhy pkwaspatchedtooutputonlyccsdspackets.theoutputlecanthen bemovedtothecomputerrunningimapp.untilnowthisisdoneviaftp, latermaybeitshouldbedonevianfs. onthisfeature.itdoesn'trunonlittleendiancomputers. NormallypkwritesEDOSheadersintheoutputles.IMAPPisusedin fromtheoperationalmodisprocessingsoftwaredevelopedatnasagoddardspaceflightcenter.itconsistofthreemaincomponents. 3.5IMAPP IMAPPstandforInternationalMODISProcessingPackageandisderived Theprogramwasimplementedonabigendianmachineandisdependent model(dem)isnecessarytoprovidealtitudeinformation.theprogram thetimeorderingofthepackets. geolocatecanalsodothegeolocationwithoutcorrectionfortheterrain CCSDSpackets.ThentheygetreformattedintoanHDFlebytheprogram L0toL1a.Thisprogramdoesafewchecksonthedata,forexampleitveries Thesecondstepisthegeolocationofthedata.Hereadigitalelevation Therstpartconvertslevel0datatolevel1abyexaminingthecontentof ampleofdataprocessingcanbedimpleforclassication(seeappendixa). correctioniskepton,becausetheoutputismoreaccurate. whichcansavealittlebitofprocessingtime.thenthedemisnotnecessary, asavingofharddiskspaceofaboutthreegigabytes.inmostcasestheterrain theproductionoftheoutputhdfleswith. InthisworkMatlabisusedtoproducearedetectionproduct. Thecalibrateddatacanbeusedforfurtherlevel2processing.Oneex- Thelaststepisthecalibrationofthedatatheprogramcalibrateand

4FireDetection Intherstpartofthissectionthealgorithmsusedfordetectinganyres aredescribedtogetherwithassociatedprocedureslikedatainputandoutput whileinthenextone(fromsection4.4onwards)theimplementationofthese 4FIREDETECTION 23 integers.thetaskistoassignaphysicalmeaningtothesevalues. DataintheHDFlesproducedbyIMAPPisstoredin16bitunsigned algorithmsisexplained. areimportant.tousethefullpossiblerangeof16bit,ascaleandaosetis used.thethirdattributestoresthephysicalunit. example,alongnameofthedatasetcanbestored.herethreeattributes 4.1AlgorithmsforPreparingFireDetection EachscienticdatasetinaHDFlecanhaveassociatedattributes.For Thesameformulaappliesforreectancevalueswiththemodicationthat canbecomputedasb=(x?o)s scales,osetoandthedataxfromthehdfleareknown,theradianceb TheunitofradiancevaluesisusuallyinWm?2m?1steradian?1.If values,theyneeddatainmeansofbrightnesstemperature.brightnesstemperatureisthetemperaturewhereablackbodywouldemitthesameamount Butmostalgorithmsforredetectiondonotworkdirectlywithradiance m2msteradian: W reectancedoesnothaveaphysicalunitassociated. wherec1rstradiationconstantc1=2hc2=1:191043910?16wm?2 ofradiationforthegivenvalue. Planck'sformulaisusedasfoundin[12]or[2]. Toconvertradiancevaluestobrightnesstemperature,theinversionof C2secondradiationconstantC2=hck?1=1:438768610?2mK Tbrightnesstemperature(K) Bradiance( m2msteradian) T= Wln(C1=(5B)+1) C2= centerwavelengthofchannel(m) hplanckconstant cspeedoflight kboltzmannconstant

4FIREDETECTION ValueMeaning 0 ShallowOcean(Oceanlessthan5kmfromcoastORlessthan50m Table3:Possiblevaluesforland/seamask 24 21354 Land(notanythingelse). OceanCoastlinesandLakeShorelines. ShallowInlandWater(InlandWaterlessthan5kmfromshoreOR Ephemeral(intermittent)Water. DeepInlandWater(Inlandwatermorethan5kmfromshoreline lessthan50mdeep). likein[12]orappendixc.2. 76IfBisstillinW=m2msteradianafactorof106hastobeinserted ModerateorContinentalOcean(Oceanmorethan5kmfromcoast DeepOcean(Oceanmorethan500mdeep). ANDmorethan50mdeepANDlessthan500mdeep). ANDmorethan50mdeep). 4.1.1EvaluationoftheLand/SeaMask Theland/seamaskisprovidedbyIMAPPina1kmresolutionandcanhave Snowhasaveryhighreectanceinmanybandsandpossibilitiesareto thevalueslistedintable3asmentionedin[22].onlypixelswithaland/sea confuserepixelswithpixelscontainingsnow.evenifsnowandiceisnot maskvalueof1(land)or2(landwithcoastline/shoreline)shouldbetested forres,allotherpixelsareignored. 4.1.2CreationandEvaluationoftheSnowMask commoninmostpartsofaustralia,thesnowymountainsarecoveredin index),butusesthebandscenteredat555nmand1640nm1insteadofred usedinthisworkarebasedon[13]. mostoftheyears. datafrommodisinstruments.itisthebasisof[17]and[13].thethresholds (NDSI)isutilized.ThisindexhasbeentestedwithdatafromLandsatand 1MODISchannels4and6 TheNDSIworkssimilartotheNDVI(normalizeddierentialvegetation Tondsnowpixelsintheimage,theNormalizedDierentialSnowIndex

andnearinfrared: 4FIREDETECTION Thesnowmappingconsistsofthreetests. NDSI=0:555?1:64 0:555+1:64 25 Thetwolattertestspreventverydarkpixelstobemappedassnow.Onlyif allthreetestsarepassedthealgorithmconsiderstheareaassnowcovered. NDSI>0:4 0:858>11%(band2) 0:555>10%(band4) taken,theywouldberegardedasres.in[1]alotofeortisundertaken anyre. 4.1.3CloudDetection Similartosnow,cloudshavelargereectancevaluesandifnocarewouldbe Theuseofthecreatedsnowmaskisthatonsnowpixelstherecan'tbe approachwaschosen. highthickclouds.similarin[4]atleasteightdierenttestfordierentcloud typesaredone.herethisdierentiationisnotnecessaryandsoadierent testaredone: toclassifydierenttypeofcloudslikethincirrus,cloudsoversnoweldsor 0:488>95%(band10) 0:531>95%(band11) 0:551>95%(band12) Asmentioned,cloudsreectmanywavelengthsverygood.Soagainthree Ifandonlyifallthreetestsarepassed,apixelisdeclaredcloudyandno simplestformcomparesthebrightnesstemperaturearound4mwithagiven redetectionisdone.thetestwasvalidatedwithdayimagesfrommodis Thereareadierentapproachestodetectresinremotesensingdata.The only. atnight)butthisisnotthecasefortheareainsouthaustralia. threshold.thisworkswelliftheremainingimageisrelativelycool(especially 4.2FireDetectionAlgorithms

thebackground.kudohdescribesin[14]theuseofnormalizeddierence indicesbetweennoaaavhrrchannels3a,2and1. betweennoaachannels3and4. 4FIREDETECTION Otherworkslike[6]usearatiobetweentheAVHRR3:7mchanneland In[16]alsotwotraditionalthresholdtestareemployedplusadierence 26 timenotavailablefornearrealtimeapplicationslikethiswork. methodsliketheneuralnetworkapproachin[18]needmuchcomputation tests,comparisonswithbackgroundsandthresholdtests. MODIS.Additionallythesexedthresholdvaryseasonally.Someother problemswiththedierenttypesecosystemfoundinlargeimagesfrom Hereaalgorithmsimilartotheonein[15]isused.Itcontainsdierence Eachtechniquehasitsdisadvantages.Fixedthresholdmethodshave thenareisassumedregardlessoftheresultsofothertests.evenifthe reonthegroundismuchhotter,theeectivebrightnesstemperaturerarely channelsusedis1km,somostresaresmallerthanapixel.secondthe reachesthisvalue.thishasdierentreasons.thespatialresolutioninthe Thesimplesttestisaxedthresholdtest.If radianceisscatteredintheatmosphereandoftenobstructedbythesmoke T3:9>360K (1) plumefromthere. butthisisonlyanintermediateresult.thenthedierencebetweenchannel 21(3:9m)andchannel31(11m) Nextthebrightnesstemperatureiscomparedtoanothervalue iscomputedandcomparedtothexedthreshold T=T3:9?T11 T3:9>325K (2) T3:9bandTbtwofurthertestsaredone WiththebackgroundvaluesT3:9bandTbandtheirstandarddeviations T3:9>T3:9b+4T3:9b T>25K (3) Thevaluesarecomparedwiththeirbackground.Ifthedierenceislarger givenlocation. thanfourtimesthestandarddeviationtheteststatesapossiblereonthe T>Tb+4Tb (4) (5)

4FIREDETECTION istruethenthepixelisdeclaredtobearepixel.soithastopassoneof Ifthefollowinglogicalcombination Test1_((Test2_Test4)^(Test3_Test5)) 27 othertests. valuesaresuggested.thevaluesof360k,25kand325kshouldbereplaced, the3:9mtestsandoneofthettests.additionally,ifitpassesthebig thresholdtest(1)itisdeclaredarepixelregardlessoftheoutcomeofall 4.3AlgorithmsusingtheResultsofFireDetection accordingto[15]with330k,10kand315k.sinceuntilnownonightpassis available,thesevaluescouldn'tbeveriedinanyway. Fornightpasseswhichmayberecordedinthefuture,otherthreshold undistortedmapwithrepixelsmarked. 4.3.1MapProjection Asexplainedinsection2.7,datafromMODISissubjecttotheBowTie Withtheresultsoftheredetectionathand,nextstepsarethevisualizationoftheseresults.Thegoalisrstalistofallresfoundandseconda dataintocommonformat,therectangularlatitude/longitudeprojectionwas eectandafewotherartifacts.toremovetheseeectsandtoputthe chosen.withthisprojectionalllatitudeandlongitudelinesareassumed introducesdistortions(aseveryprojectionfromaspheretoaplanedoes,see [27])intotheimage.Thefurtherapartfromtheequator,themoreexpanded parallelandofequaldistance.ofcoursethisisnottruesothisprojection thedataget.tobalanceoutthiseectalittlebitthefollowingassumptions aremade: Thedistancebetweenlongitudelinesis Thedistancebetweenlatitudelines(setofpointswiththesameinteger sucientforthisapplication. latitudevalue)is111.2km.thisnotexactforeverypointonearthbut waschosentobe34.86degreessouthasthisisthepositionofthe sucientaccuracyhere. receivingdishinmawsonlakes/adelaide. fortheentireimage.thisalsoassumesasphericalearthbuthasa 111:2kmcos

partsnorthofadelaideandexpandedforpartssouthofadelaide.another Theseassumptionshavetheconsequencethattheimagegetscompressedfor 4FIREDETECTION Thesizeofthesmallestpixelsinthenominal1kmresolutionisexactly 1kmx1km.Thisisusuallyonlytrueforpixelsalongthenadirline. 28 resultoftradingallimageswiththesameassumptionsisthattheseimages foundinthegeolocationdata.eachpixelintheconstructedgridhasa canbeconstructedbasedonthemaximumlatitudeandlongitudevalues canttogethertoformabiggermapifdesired. andawidthof heightof Withthenowknowndistancebetweenlatitudeandlongitudelinesagrid (111:2km deg)?11km (111:2km deg)?11km pixelcos(adelaide)=0:010959deg pixel=0:0089928deg 4.3.2Smoothing Thesevaluesareconstantforallimages.Themaximumshiftbetweentwo thisonepixel(orless)shiftdierentimagesttogether. dierentimagesisonepixelbecauseofdierentimageorigins.despiteof pixel: viewableimage. remainblack.theseblackspotshavetobelledwithdatainordertogeta imagestayblackbecausetheimageelementsizeattheleftandrightedge recordedelementgetsprojectedinthenalgrid,manyspotsinthisgrid ismuchlargerthan1km.infactaimageelementcangrowbasedonthe 55degreemaximumviewangleofMODISupto5x2km.Sinceonlytheone Thedrawbackofthedescribedprojectionisthatmanypixelsintheresulting eachpixeltobelledthemedianlterconsidersallitsneighborsaccording approachwastheuseofmedianrankorderingltersforeachcolorband.for Forlargerltersizesthisapparentlyintroduceserrorsbecausepixelsfaraway totheltersize,sortstheminvalueorderandselectstheoneinthemiddle. thenumberofcandidatesoftengetseven(like2).sincethelternowcan't fromtheonetobelledcanlargelyinuencetheresult.forsmallltersizes Twodierentwaysofllingtheseblackpointshavebeentried.Therst value1)ortoobright(iftheltertriesvalue2inthiscase). takethemiddlevaluetheimagesgetseithertoodark(ifthelterchoose sumofallnonzeropixelsintheltersizeisdividedbytheircount.the Thesecondvariantofpixelllingwaswithaaveraginglter.Herethe

Thebestresultshavebeenfoundwithanaveraginglterofsize3x3applied darkpixelsremain. (likeclouds)spreadoveralargearea.ifasmallerltersizeischosen,many disadvantageoflargerltersizesislikeabovetheinuenceofextremepixels 4FIREDETECTION Theremainingblackpixelscanbeeliminatedbyreapplyingthelter. 29 twice.smallersquareshapedltersizesdonotmakesensesincea1x1lter Thenextlargeroddsizedlter(5x5)alreadyintroducesmanymoreerrors. doesn'tchangeanythinganda2x2ltershiftstheimageinonedirection. 4.4ImplementationofFireDetection FortherealizationofthedescribedalgorithmsMatlab6Release12wasthe chosenlanguage.thishasseveralreasons: AlgorithmsareeasytoformulateinMatlab.Thewrittenprogramsare Matlabhasagoodsupportforallneededleformats,particularlythe Theresultingprogramsarehighlyplatformindependentandcanrun easiertounderstandthansimilarprogramsinforexamplec. inputformathdfandtheoutputformattiff. ManycommonexceptionsarecaughtbyMatlab.ForexampleadivisionbyzeroisnotuncommonwhilecomputingtheNDSI.Thiscauses Theexecutionspeedisverygoodwhiledoingmatrixoperationsand oneveryplatformsupportedbymatlab. sucientinallothercases. in64bitdoublevalues.thisisespeciallymemoryinecientfor1bitlogical arrays. AdisadvantageofMatlabisitsdemandtostoreeverydataforcomputations andnotaruntimeerrorwithprogramabort. AllcomputationsfromtheendoftheIMAPPrunaremainlydoneinone onlyasuppressiblewarningmessageandtheresultnan(notanumber) Thefunctionconsistsofthefollowingparts: oneingure5.therethemainpartsoftheimplementationcanbeseen. Matlabfunction.Theaimofthissectionistodescribetheimplementation firedetectioninsomedetails. ofthealgorithmsmentionedaboveandtherunningofthematlabscript 1.Readinginallnecessarydata,i.e.importingpartsoftheHDFles Asamplerunofthescriptproducesoutputwhichlookssimilartothe

4FIREDETECTION 4.computationofthemap 3.Theactualredetection 2.Executingsanitycheckslikedetectionofcloudsandwater 30 itsexecution.theentiresourcecodeofthescriptisinappendixc.1. isstoredinhdf(hierarchicaldataformat)les.forabetterdescription oftheleformatseesection2.6.3and[25].sincematlabhasanexcellent 5.Andnallytheoutputoftheresults. supportforthehdfformat,thefunctionsforreadingdata(sdsread.min appendixc.4)andassociatedattributes(attributeread.minappendixc.3 Nowfollowsamoredetailedlookatthesourcecodeofthefunctionand canbekeptshort. Therstmaintaskistheimportofnecessarydata.DatafromIMAPP matetheconversionfromradiancetobrightnesstemperaturethefunction perature,butvaluesinthehdflesarestoredasradiances.toauto- land/seamaskareread.errorcheckingisdonewhereapplicable. 11,12,21and31.Additionallythemetainformationlatitude,longitudeand tionisnecessary.sothescriptreadsinthemodisbands1,2,3,4,6,10, Thealgorithmforredetectionneedsdataintermsofbrightnesstem- Forthedierentalgorithmsanumberofbandsandsomemetainforma- whereradianceisexpectedinw=m2msteradianasfoundinthehdf result=radiance2teff(radiance,lambda) radiance2teff.mwaswritten.callingsyntaxis viahelpradiance2teffissupported. canbeconvertedatonce. entlybaddata.thisincludesreectancevaluesgreaterthanone,brightness turnedinkelvin.theparameterradiancecanbeamatrix,somanyvalues lesandlambdashouldbeinm.aftersuccessfulcompletionresultisre- temperaturesgreaterthanthesaturationvaluesofthesensor,valueswhere radiance2teffcomputedinvalidresultsorwherethebaddatamarkerwas insertedbyimapp(-999forlatitude/longitudevalues). Thenextstepintheredetectionscriptistheeliminationofappar- ThesourcecodeisfoundinappendixC.2.TheusualMatlabhelpfacility ofimplementingsuchalgorithmsinmatlabcanbeseeninlineslike onlyvaluesof1or2areregardedaspossibleplacesforres.theelegance whichcreatesalogicarrayofthenecessarysizecontainingtrueforallland pixelsandfalseforallothers. land=(landseamask==1 landseamask==2); Theevaluationoftheprovidedland/seamaskisdoneasdescribedabove,

4FIREDETECTION 31 readingfinished readdatafromdisk >>firedetection(filename); metainformation 1kmemissive 1kmreflective 250mreflective 500mreflective Figure5:Sampleoutputfromthescriptfiredetection convertingradiancestotemperaturevalues lookingforbaddata evaluatingland/seamask detectingsnow detectingclouds detectingfires eliminatingbaddata 4um 11um creatingrgbimage producingrectangularmap smoothing(1/2) computingbackgrounds testing finding numberoffirepixelsdetected:78 calculatingvectorsandcopyingdata onedegreelongitudeis91.2453pixelswide onedegreelatitudeis111.2pixelshigh writingimage smoothing(2/2) findingblackspots countpixelswehavegot filtering writingasciioutput alldone >> paintingfires red green blue

thenthereisdenitelynosnowatthispoint.additionallyoneoftheother ariseswhenbothvaluesarezero.thenthedivisorbecomeszeroanda divisionbyzerowarningoccurs.thiswarningcanbeandissuppressed. Theinvalidresultisnoproblembecausewhenbothbandvaluesarezero 4FIREDETECTION Aproblemduringsnowdetection(infactduringcomputingtheNDSI) 32 twotestsfailstoo. Thisisachievedwiththelter lter.theeectisasmothercloudmaskincludingcloudedges. isaarrayoflogicvalues,theltercouldbedescribedasa3by3logicor cloudmaskisbroadenedbyonepixel.thisisdonebyconvolutingthecloud maskwitha3by3matrixcontainingonlyones.becausethecloudmask Fortheredetectionthecomputationofthepixelsbackgroundisneeded. Theclouddetectionworksstraightforwardexceptattheend,thefound 140 0 B@ andthenbackgroundvaluesarecomparedtothepixels.allthresholdtests theedgesareexcludedfromfurtherprocessing. Theproblemwithconvolutionltersisalwaysattheedges.That'swhy Thestandarddeviationofeachbandiscomputedforthewholeimage 1100011 1111111 1 CA aredonewithdaytimevalues.anighttimetestwasnotdoneatall. sitionsinthenalmap.thisisthetaskofthefunctioncstretch,which angularlatitude/longitudemap. valuesisdoneinthesubfunctionstretch. inordertodisplaythem.tofurtherimprovetheimage,aclippingofhigher ThelatitudeandlongitudevalueshavetobeconvertedtoXandYpo- Therestofthefunctiondealsmainlywiththecomputationoftherect- takesthegriddingfactorasaninput.thegriddingfactoris111.2kilometer forlatitudevaluesand111:2kmcos(adelaide)forlongitudevalues.when Thevaluesinthevisiblebandshavetobecompressedintothe83bit map,theimageiswrittentodiskandthereportingtextleisgenerated.at ingpixelsarelledwithaveragesoftheirneighbors. themap. theconversionisnished,thered,greenandbluepixelvaluesarecopiedinto Notallpixelsinthenalmapgetlledwiththisprocedure.Theremain- Afterthedetectedrepixelsaremarkedwithredspotsinthesmothed

thispointthefunctionreturns. 4FIREDETECTION 33

5CONCLUSIONS doesallworkfromtherecordedrawletotheredetectionproduct. thenecessarysequencebythesmallshellscriptinappendixe.3.thisscript 5Conclusions Allprogramsmentionedandallalgorithmsdescribedarecalledandrunin 34 sionwasmadetosimplifytheprocessandkeepallcomputationononeplace. inanautomaticsystem.becauseslicereliminatesmostofthebaddata Additionallyitproveddiculttocoordinatethemanydierentprogramsof already,thelackoferrorcheckinginframesyncisnotthatadverse. thestpsystem,sometimeskeystrokesarenecessary.thisisnotdesirable InthisscripttheprogramFrameSyncisusedinsteadofSTPS.Thisdecidatedinanumberofways: Atrstavisualinspectionoftheresultingmapwasdone.Allres Theresultsoftheredetectionalgorithmshavebeenempiricallyvali- foundareonlandandnotobstructedbyclouds.allreswerenear NextstepwasthecomparisontosimilarpicturespublishedbyNASAin eitheravisiblesmokeplumeoravisiblerescar.howeversomesmoke lutionofmodis.insomecasesthesmokeplumeitselfprohibitsthe plumesdon'thavearepixelnearthem.thishasseveralreasons. Someresarejusttoosmalltobepickedupbythe1kmx1kmreso- [24].Sincetheavailableimageswerefourdaysapart,onlyaveryrough detectionofthecausingrebyeitherbeingdetectedasacloudorjust obstructingtheviewtotheground. LastthedatawascomparedtothedatapublisheddailybyDOLA[9] comparisonwaspossible,butitshowedasimilarcongurationofre orignited. pixels.somereshaveadvancedadistance,otherswereextinguished Allvalidationsdoneshowtheresultsareveryreasonable.Thelackofmore datapreventedamorestatisticalevaluationofthecorrectnessofthealgorithm. anothersensor.astatisticalanalysiswasnotdonebecauseoflackof res.thecomparisonlookspromisingandshowsmanysimilarresults. ontheirwebsite.therethenoaa-avhrrdataisusedtodetect Asampleoutputoftheworkcanbeseeingure6.Thefoundresin enoughdata. Dierencesarecausedbythedierentrecordingtimeandtheuseof thisgurearelisted(amongothers)ingure7.theimagewasrecorded on10/08/2001andshowsanareaneargulfofcarpinteriainthenorthern Territory.Firescarsandsmokeplumescaneasilybespotted.

5CONCLUSIONS 35 Figure6:Pictureofdetectedres

5CONCLUSIONS #inputfiles: #ResultsoffiredetectionFigure7:Listofdetectedres 36 #startingtime:25-nov-200111:31:18 #/dd1/imapp_run/ter10aug58-174.fires.txt(thisfile) #outputfiles: #/dd1/imapp_run/ter10aug58-174.fires.tiff #finishingtime:25-nov-200111:33:46 #/dd1/imapp_run/ter10aug58-174.1000m.hdf #/dd1/imapp_run/ter10aug58-174.geo.hdf #thefollowingfireswerefound: #latitudelongitudeposit-xposit-y -16.87312135.21036-16.87901135.21593 pixelhigh:0.0084998degreeslatitude pixelwidht:0.016104degreeslongitude imagerange:-20.9236/133.1496to-14.3969/155.9702-16.88553138.45078-16.88549138.44351-16.88902138.46284-16.88893138.47014-16.89471138.42897-16.89810138.44839-16.89806138.44115-16.90503135.20378-16.90158138.46049 291 292 293-16.90147138.46777-16.90994138.57751-16.90902138.58495 294-16.91895138.55623-16.91815138.56363-16.92238138.57518 295 328-16.92145138.58260-16.97014138.40166 296 329-16.97005138.40747 128-16.97369138.42108-16.97346138.42691 330-17.05182141.13303 297-17.05598141.09351 338-17.74360136.74628-17.75962136.76088 336-17.76906136.73309 303-17.78312136.73033 312 337-17.78768136.75540 313-17.79222136.78035 394 326-17.79673136.80521 396 327-17.81071136.80246 397 328-17.96402136.60155 398 496-17.96860136.62700 493-17.97316136.65234 399-18.14500138.27626 400-18.14830138.29526 402 223-18.15743138.27394 222-18.16072138.29294 420 224-18.16398138.31187 421 441 442 443 225 227 214 216 218 320 318 319 321

6Recommendation Thereareanumberofeldswherefurtherworkcanbedone.Theseinclude 6RECOMMENDATION Attempttoforecastreintensitylikein[21] 37 Themappinghasdisadvantages.Especiallywhenatemporalanalysis Itispossibletodetecttheamountofsmoketodrawconclusionsabout Firescardetectionisusefulfortheestimationoftheamountofburnt theres.zhanqingliet.al.suggestedin[18]theuseofneuralnetworks todistinguishbetweensmokeandclouds. isdesired,amoreaccurategriddingofthedataisnecessary.works like[28]showhowthiscanbedone. Theprogramslicer.cshouldbemodiedtosliceevenwhennobad AVHRRdata. dataisfoundatcertainlesizeslikeevery400megabyte.thiswould areaandtheamountofproducedco2.in[5]thisisshownwith WhenAquawillbelaunchedafewadjustmentsmustbedonetothe Furthervalidationmustbedone.EspeciallyDOLA[9]shouldbea FrameSyncprogramtoadoptchangesin preventsomelackofmemoryproblemsandspeedupprocessing.additionallythestorageofimagesmaybeeasier. goodresourceforthiswork. Processingofnightimageswasnottriedatallandmaypromisegood resultsinredetection(butnotinimagery).forthistoworksomeminorchangestoframesyncarenecessary.thefunctionfiredetection {anddierentspacecraftid. {sequenceinpnandrsdecoding alsohastobechangedabitbecausesomebandsarenotavailableat night.

ADIMPLE DimpleisadigitalimageprocessingsystemfromProcessSoftwareSolutionWollongong,Australia.Itisdesignedasageneralimagemanipulation andprocessingsystembuthassomeinterestingfeaturesspeciallyforremote 38 ADimple togreyscaleimages,3dplotsordierentimageratios.sincenormalized ratiosaresupported,thelatteronecanbeusedforproducingindiceslike NDVIorNDSI. sensingandsatelliteimagery. lutionandrankorderlters.specialltersareforexamplethede-striping ltersforsatelliteimages. Theimageenhancementparthasthestandardltersincludingconvo- ThegeneralfeaturesincludesomeimagetransformationslikefromRGB IOLwhichisasimpleprogramminglanguageforimagemanipulation. allwithgroundcontrolpointswhichhavetobesuppliedbytheuser.this isamanualprocessandwastherefornotseenasanoptionforthisproject. manually. Thesameistruefortheclassicationalgorithms.Theyhavetobetrained Thebuilt-inlterscanbeenhancedwiththeImageOperationLanguage Therectication,registrationandresamplingalgorithmsprovidedwork IMAPP.ToimporttheselesinDimple,Dimplehastoknowhowtoread them,sotheuserhastosupplyforexampletheresolutionoftheimageand HDFlesandexporttheminaformatDimplecanread.AppendixD.2 containsthesourcecodeofthismatlabfunction.dataisexportedinbinary ofdimpletoreadhdfles.toovercomethismatlabwasusedtoimport lescontainingnootherinformationthantheactuallycalibrateddatafrom ThemaindisadvantageinconjunctionwithMODISdataistheinability thenumberofbandstoimport. hassuppliedthisinformation.forexampleitispossibletotransferlatitude/longitudedataasrealnumbersandintegerdataasuint16. Dimplecanreadsuchbinaryleswithdierentdatatypesaftertheuser

BGLOSSARY BGlossary ASMAttachedSyncMarker;beginningofeveryCADU ASTRAAutomaticSatelliteTrackingResearchAntenna;receivingfacility 39 AVHRRAdvancedVeryHighResolutionRadiometer;Instrumentonboard CADUChannelAccessDataUnit;consistsofASMandCVCDU ByteEightbits,usedsynonymouslywithoctet. initr CCSDSCenterforComputationalScienceandAdvancedDistributedSimulation NOAAsatellites CVCDUCodedVirtualChannelDataUnit;consistsofVCDUandReed HDFHierarchicalDataFormat;usedfordatastorageinlevel1aandlevel1b DEMDigitalElevationModel ITRInstituteforTelecommunicationsResearch MODISModerateResolutionImagingSpectroradiometer;Instrumenton SolomonCode NDVINormalizedDierenceVegetationIndex NDSINormalizedDierenceSnowIndex NCSANationalCenterforSupercomputingApplications MPDUMultiplexedProtocolDataUnit boardterraandaquasatellites NOAANationalOceanicandAtmosphericAdministration;partofU.S. OctetEightbits,usedsynonymouslywithbyte. STPSSatelliteTelemetryProcessingSystem;SoftwarefromGSFC VCDUVirtualChannelDataUnit;transportentityforCCSDSpackets wordsixteenbitsofdata DepartmentofCommerce

CMATLABSOURCEFILESFORFIREDETECTION 40 CMatlabSourceFilesforFireDetection InthissectionallMatlablesusedwhileredetectionarelisted.Some additionalfunctions,whicharenotnecessarybutsometimeshelpful,canbe seeninappendixd. C.1firedetection.m functionf=firedetection(filename); %FIREDETECTIONdetectfiresusingdatain`filename` %firedetectionisalongerscript %fordocumentation,pleasereferto %thesourcecodeand %KendyKutzner: %"ProcessingMODISDataforFireDetectioninSouthAustralia", %2001,ITR,Adelaide %changehistory %08/2001to10/2001writtenbykendykutzner %11/2001documentedbykendy %12/2001speedupbyvectorizingmappingbykendy %12/2001furtherdocumentationbykendy %let'sstart starttime=now; %nofailureuntilnow %mostcommonerrorsarecatched. %ofcoursenotall. %probablythethemostseriuosnotcatched %errorisoutofmemory.inthiscasematlab %abortsthefunction.thisscripttriestoclearmemory %asearlyaspossible. f=0; %readallthedata disp('readdatafromdisk'); %atfirst,weneedsomeinformationaboutthedatain %the.1000m.hdffile disp('metainformation'); filename1=[filename'.geo.hdf']; %forobviousreasonsweneedthese: [latitude,stat1]=sds_read(filename1,'latitude'); [longitude,stat2]=sds_read(filename1,'longitude'); %therearenofiresonwater,somakeuseof %theland/seamask [landseamask,stat3]=sds_read(filename1,'land/seamask'); %tohavebettervisualcontrolwhenneededall %dataistransposedsoitcanbeviewedwith %imagesc(bandx) %additionally,theimwrite()functionwantsthe %datainthisdirectiontoproduceimagesinthe %'right'direction latitude=double(latitude'); longitude=double(longitude'); landseamask=landseamask';

CMATLABSOURCEFILESFORFIREDETECTION 41 %catchsomeerrors if(stat1~=0 stat2~=0 stat3~=0) f=-1; disp('can''treadlatitude/longitudeorland/seamask'); return; end %nowwereadinallthedatabandsweneed. disp('1kmemissive'); filename1=[filename'.1000m.hdf']; bandname='ev_1km_emissive'; [emissives,stat1]=sds_read(filename1,bandname); [scales,stat2]=attribute_read(filename1,bandname,'radiance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'radiance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1; disp('can''treademissivebands'); return; end offsets=double(offsets); scales=double(scales); %datain.hdffilesisstoredwithscaleandoffset %tomakebetteruseof16bitdatafields. %Weneedtheactualvaluessoconvertthem. band21=((double(emissives(:,:,2))-offsets(2))*scales(2))'; %transposethearraysforthereasonabove band31=((double(emissives(:,:,11))-offsets(11))*scales(11))'; %thesearraystendtobequitebig,sofreememoryifwecan clearemissives; %iflackofmemoryisstillaproblem,a'pack'statementcan %beinsertedonthecriticalpoints. disp('1kmreflective'); filename1=[filename'.1000m.hdf']; bandname='ev_1km_refsb'; [reflectives,stat1]=sds_read(filename1,bandname); %forreflectivevalues,differentscalesandoffsetsapply [scales,stat2]=attribute_read(filename1,bandname,'reflectance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'reflectance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1; disp('can''treadreflectivebands(1km)'); return; end offsets=double(offsets); scales=double(scales); band10=((double(reflectives(:,:,3))-offsets(3))*scales(3))'; band11=((double(reflectives(:,:,4))-offsets(4))*scales(4))'; band12=((double(reflectives(:,:,5))-offsets(5))*scales(5))'; clearreflectives; disp('250mreflective'); filename1=[filename'.1000m.hdf']; bandname='ev_250_aggr1km_refsb'; [reflectives,stat1]=sds_read(filename1,bandname); [scales,stat2]=attribute_read(filename1,bandname,'reflectance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'reflectance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1;

CMATLABSOURCEFILESFORFIREDETECTION 42 disp('can''treadreflectivebands(250m)'); return; end offsets=double(offsets); scales=double(scales); band1=((double(reflectives(:,:,1))-offsets(1))*scales(1))'; band2=((double(reflectives(:,:,2))-offsets(2))*scales(2))'; clearreflectives; disp('500mreflective'); filename1=[filename'.1000m.hdf']; bandname='ev_500_aggr1km_refsb'; [reflectives,stat1]=sds_read(filename1,bandname); [scales,stat2]=attribute_read(filename1,bandname,'reflectance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'reflectance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1; disp('can''treadreflectivebands(500m)'); return; end offsets=double(offsets); scales=double(scales); band3=((double(reflectives(:,:,1))-offsets(1))*scales(1))'; band4=((double(reflectives(:,:,2))-offsets(2))*scales(2))'; band6=((double(reflectives(:,:,4))-offsets(4))*scales(4))'; clearreflectives; clearscalesoffsetsbandnamefilename1 %hopefullynowwegotallthedataweneed disp('readingfinished'); %correctnessofvaluesuntilnowverifiedwithhdflook %fromnowon,weareonourown disp('convertingradiancestotemperaturevalues'); %calculatethemiddleofthechannelsonthefly disp('4um'); band21=radiance2teff(band21,(3.989+3.929)/2); disp('11um'); band31=radiance2teff(band31,(10.780+11.280)/2); disp('lookingforbaddata'); %iftheconversionaboveproducedcomplexvalues, %theargumenttolog()wasnegative.something %isseriouslywrong,wewanttogetridofthis. imags=imag(band21) imag(band31); %the4umchannelsaturatesat500k,the11umchannelat400k %soallvaluesabovearebad %sametoreflectivebands:valuesabove100%arebad baddata=find(band1>1 band2>1 band3>1 band3>1 band4>1... band6>1 band21>500 band31>400 imags... latitude==-999 longitude==-999); clearimags; disp('eliminatingbaddata'); fori=[123461011122131] istr=num2str(i); evalstring=['band'istr'(baddata)=0;']; eval(evalstring); end %specialcaseforlat/long,becauseweneedthe %min()andmax()functionlater.ifwewouldinsert %zeros,oneofthemwouldgetconfused. %min()andmax()takecareofnans,sowecanuseit

CMATLABSOURCEFILESFORFIREDETECTION 43 latitude(baddata)=nan; longitude(baddata)=nan; disp('evaluatingland/seamask'); %wewantonly"land"or"landwithcoastline" land=(landseamask==1 landseamask==2); clearlandseamask; disp('detectingsnow'); %nextlinemayproducedivisionbyzerowarning. %thisisok,soturnitoff warningoff; ndsi=(band4-band6)./(band4+band6); %wewanttohearwarningsagain warningbacktrace; %thisisthesnowdetectionalgorithm snow=((ndsi>.4&band2>.11)&band4>.1); snowpos=find(snow&land); %nexttotheclouds disp('detectingclouds'); cloud=((band11>0.95*max(band11(:)))&... (band10>0.95*max(band10(:)))&... (band12>0.95*max(band12(:)))); %theaboveclouddetectionalgorithmisnotthebest %Togetallthecloudedges,webroadenthecloudmask %withasmoothingfilter cloud=conv2(cloud,ones(5),'same'); %bringcloudmasktologicarrayagain cloud=cloud&1; cloudpos=find(cloud&~(snow&land)); %nowtothemainpart disp('detectingfires'); %atfirstweconstructabackgroundfilter backgroundfilter=ones(7); %weareonlyinterestedinthesurrounding backgroundfilter([345],[345])=0; %normalizethefiltersoitdoesn'tdisturbdata backgroundfilter=backgroundfilter/sum(backgroundfilter(:)); disp('computingbackgrounds'); %itprovedconvinienttomarkbaddatawithnan band21(band21==0)=nan; band31(band31==0)=nan; t41=band21-band31; t4b=conv2(band21,backgroundfilter,'same'); t41b=conv2(t41,backgroundfilter,'same'); %byfiltering,theedgeofeachmatrixbecomesinvalid %soweconstructainvalidmask invalid=zeros(size(t4b)); invalid([1:6,size(invalid,1)-5:size(invalid,1)],:)=1; invalid(:,[1:6,size(invalid,2)-5:size(invalid,2)])=1; %nowwecanseewhynanwasthebetterbaddatamarker invalid=invalid isnan(t4b) isnan(t41b); %wecomputethestandarddeviationonlyforvaliddata dt4b=std(t4b(~invalid)); dt41b=std(t41b(~invalid)); %nextistheperformingofthedifferentfiretests disp('testing'); day=1;

CMATLABSOURCEFILESFORFIREDETECTION 44 night=2; %thenextlinehastobereplacedbytheactual %day/nightdetection time=day; %thethresholdvaluesarebasedonatbd-mod-14 threshold=[33025360;31510330]; %nowthefivetests firetest1=(band21>t4b+dt4b*4); firetest2=band21>threshold(time,1); firetest3=(t41>t41b+dt41b*4); firetest4=t41>threshold(time,2); firetest5=band21>threshold(time,3); disp('finding'); %combiningofthetests fire=(((firetest1 firetest2)&(firetest3 firetest4))... firetest5)&~invalid; %firescanonlybeonland,withnoclouds,andnosnow firepos=find(fire&land&~snow&~cloud); %theactualfiredetectionisnowdone. %restofthescriptdealswithmapconstruction disp(['numberoffirepixelsdetected:'num2str(size(firepos,1))]); %testingisdone,sowecancleanupalittlebit clearfiretestafiretestbfiretestafiretestbfiretestc; cleart41t41bt4bdt4bdt41b; clearband2band6band10band11band12band21band31; clearfirecloudsnowbaddatainvalidlandlandseamaskndsi; %nowproduceoutputimage disp('creatingrgbimage'); rgb=stretch(cat(3,band1,band4,band3)); %thefollowinglineswereusedfordebugging. %Iftheyareturnedonagain,theyproduceanice %imagetoo. %imagesc(rgb); %rgb2=rgb; %rgb2(firepos)=1; %rgb2(firepos+size(rgb2,1)*size(rgb2,2))=0; %rgb2(firepos+size(rgb2,1)*size(rgb2,2)*2)=0; %rgb2(snowpos)=0; %rgb2(snowpos+size(rgb2,1)*size(rgb2,2))=0; %rgb2(snowpos+size(rgb2,1)*size(rgb2,2)*2)=1; %rgb2(cloudpos)=0; %rgb2(cloudpos+size(rgb2,1)*size(rgb2,2))=0; %rgb2(cloudpos+size(rgb2,1)*size(rgb2,2)*2)=0; %figure %imagesc(rgb2) %continuewiththerealstuff disp('producingrectangularmap'); disp('calculatingvectorsandcopyingdata'); %wedon'twanttoloosedata,sointhemiddleofthe %mapapixelshouldbe1kmx1km. %onedegreelatitudeshouldbe111.2kmeverywhere %sizeofonedegreelongitudeisdependentonlatitude %weuseaustralia/adelaide/mawsonlakes ourlatitude=-34.86; %the'0-'inthenextlineistherebecauselatitudesare

CMATLABSOURCEFILESFORFIREDETECTION 45 %countedfromsouthtonorthwhilematlabimagesfromtop %tobottom [nlat,latfac]=cstretch(0-latitude,111.2); disp(['onedegreelatitudeis'num2str(latfac)'pixelshigh']); %cos(latitude)istheshrinkingfactorforlongitudevalues %thisassumesasphericalearthwhichisperfectlyrightfor %thispurpose [nlong,longfac]=cstretch(longitude,111.2*cos(ourlatitude/180*pi)); disp(['onedegreelongitudeis'num2str(longfac)'pixelswide']); %matlabindicesstartwith1 nlat=nlat+1; nlong=nlong+1; %createthetargetmatrixtospeedupcopying recmap=zeros(max(nlat(:)),max(nlong(:)),3); %thenextlinescopythedatafromrgb(:,:,:)to %recmap(:,:,:)accordingtotheinformation %infromlatitude/longitude vector=sub2ind([size(recmap,1)size(recmap,2)],nlat(:),nlong(:)); recmap(vector)=rgb(:,:,1); recmap(vector+size(recmap,1)*size(recmap,2))=rgb(:,:,2); recmap(vector+size(recmap,1)*size(recmap,2)*2)=rgb(:,:,3); %becausenotallpixelsareofthesamesize,inrecmap() %areblackspots.wewanttofillthem. %thisoperationwillneedsomememory,socleanupagain clearvector clearcloudposrgbsnowpos clearband1band3band4 pack; disp('smoothing(1/2)'); recmap=rgbfilter(recmap); disp('smoothing(2/2)'); recmap=rgbfilter(recmap); %thecodetopaintalinegridmaybeinsertedhere %nextthingistomarkthefireswithbigredpixels disp('paintingfires'); nfp=cat(2,nlat(firepos),nlong(firepos)); fori=1:size(nfp,1) recmap(nfp(i,1)+1,nfp(i,2)+1,:)=[100]; end %ifyouwanttoseetheresult... %figure; %imagesc(recmap); %disp('writingimage'); %imwritetendstousealotofmemory %soclearupandconvertthedatabeforehand clearband1band3band4rgbrgb2cloudpos; recmap=uint8(round(recmap*255)); pack; imwrite(recmap,[filename'.fires.tiff'],'tiff'); %hereweare!

CMATLABSOURCEFILESFORFIREDETECTION 46 finishtime=now; %lastthingsistowritethereport %ifotherinformationareneededinthereport %itiseasytomodifyit disp('writingasciiouput'); msg=['#resultsoffiredetection'10... '#inputfiles:'10... '#'filename'.geo.hdf'10... '#'filename'.1000m.hdf'10... '#outputfiles:'10... '#'filename'.fires.tiff'10... '# imagerange:'num2str(min(latitude(:)))'/'... num2str(min(longitude(:)))... 'to'num2str(max(latitude(:)))'/'... num2str(max(longitude(:)))10... '# pixelhigh:'num2str(1/latfac)... 'degreeslatitude'10... '# pixelwidht:'num2str(1/longfac)... 'degreeslongitude'10... '#'filename'.fires.txt(thisfile)'10... '#startingtime:'datestr(starttime)10... '#finishingtime:'datestr(finishtime)10... '#'10... '#thefollowingfireswerefound:'10... '#latitudelongitudeposit-xposit-y'10... ]; %fopen()shouldnotfail.ifitfails,wecan'tdoanythinganyway fid=fopen([filename'.fires.txt'],'w'); fwrite(fid,msg,'char'); fireinfo=cat(2,latitude(firepos),longitude(firepos),... nlat(firepos),nlong(firepos)); ifisempty(fireinfo) fwrite(fid,'#none','char'); elsefireinfo=sortrows(fireinfo,[3412]); fprintf(fid,'%10.5f%10.5f%10.0f%10.0f\n',fireinfo'); end fclose(fid); %ifwereachthispointwearehappy! disp('alldone'); return; %----- %somehelpfulinternalfunctions %stretchingcolorinthergbimage functionres=stretch(inp); mi=min(inp(inp~=0)); tmp=inp-mi; ma=.3*max(tmp(:)); res=tmp/ma; res(res>1)=1; res(res<0)=0; return; %stretchinglat/longvaluestomatrixcoordinates function[res,factor]=cstretch(inp,s);

CMATLABSOURCEFILESFORFIREDETECTION 47 secondfactor=1; tmp=inp-min(inp(:)); factor=s*secondfactor; tmp=round(tmp*factor); tmp(isnan(tmp))=1; res=tmp; return; %thefilteringoutoftheblackpixels functionrecmap=rgbfilter(recmap); fs=1;%sizeofthefilter xy=size(recmap); disp('findingblackspots'); %firstweneedtofindwhichpixelscanbefilled recmapcopy=sum(recmap,3); recmapcopy=conv2(recmapcopy,ones(2*fs+1),'same'); %excludetheedgesagain%%necessary?? recmapcopy([1:fs+1xy(1)-fs-1:xy(1)],:)=0; recmapcopy(:,[1:fs+1xy(2)-fs-1:xy(2)])=0; cxy=find(sum(recmap,3)==0&recmapcopy~=0); %allcoordinatesincxycanpotentiallyfilled %withdata disp('countpixelswehavegot'); %howmanynonzeroneighborshasapixel? %weneedthisasadivisorintheaveraging divi=zeros(size(recmapcopy)); clearrecmapcopy a1=(2*fs+1); %inthenextlinethe'&1'convertsthedoublearray %inalogicalarrayofonesandzeros. %wethensumovertheseones. divi=conv2((recmap(:,:,1)&1),ones(a1),'same'); disp('filtering'); disp('red'); tmp=conv2(recmap(:,:,1),ones(a1),'same'); recmap(cxy)=tmp(cxy)./divi(cxy); disp('green'); tmp=conv2(recmap(:,:,2),ones(a1),'same'); recmap(cxy+xy(1)*xy(2))=tmp(cxy)./divi(cxy); disp('blue'); tmp=conv2(recmap(:,:,3),ones(a1),'same'); recmap(cxy+xy(1)*xy(2)*2)=tmp(cxy)./divi(cxy); return; C.2radiance2teff.m functionteff=radiance2teff(radiance,lambda); %RADIANCE2TEFFconvertradiancevaluestobrightnesstemparatures %inputvalues: %radianceinw/m^2/um/sterradasfoundinimapphdffiles %lambdainum %radiancecanbeamatrix %ouputvalues: %teffeffectivebrightnesstemperatureinkelvin %changehistory

CMATLABSOURCEFILESFORFIREDETECTION %MODISAirborneSimulatorResearchandDocuments %??/11/2001addedcomputationofc1andc2 %citinghttp://ltpwww.gsfc.nasa.gov/mas/masdug.html %??/11/2001documented %08/11/2001writtenbykendykutznerfromscratchusingcitationbelow 48 %T(L,B)=C2/L*loge(C1/(L5*B(L,T)*106)+1) %where, %equation.the %inverseequationisoftheform %or'brightnesstemperature'maybedonebyinvertingtheplanck %ConversionfromIRradiancetoPlanckequivalenttemperature %DataUserGuide %byliamgumley,paulhubanks,andedmasuokaapril1994 %T(L,B)=brightnesstemperatureindegreesKelvin, %C2=(h.c)/k=1.4387686.10-2mK %B(L,T)=PlanckradianceinWm-2sr-1um-1 %l=wavelengthinmeters %C1=2.h.c2=1.1910439.10-16Wm-2 %speedoflightinvacuum %Planckconstant h=6.62606876e-34; %secondradiationconstant c2=h*c/k; %firstradiationconstant c1=2*h*c^2; %c2=1.4387686e-2; %c1=1.1910439e-16; c=299792458; %Boltzmannconstant radiance=radiance.*1e6; k=1.3806503e-23; %voila! C.3attributeread.m function[res,f]=attribute_read(filename,sdsname,attrname); %bringwavelengthtom teff=c2./lambda./log(c1./(lambda^5.*radiance)+1); %bringradiancetow/m^2/m/sterrad lambda=lambda./1e6; %filenameisthenameofthe.hdffiletoopen.itisnotparsedin %anyway,soitmaycontainpathinformationanditmustcontain %[res,status]=sds_read(filename,sdsname,attrname) %asinthe.hdffile. %theextension.hdf %sdsnameisthenameofthedatasettobeopened.itmustappearexactly %ATTRIBUTE_READreadsinScientificDataSetsfrom.HDFfiles

CMATLABSOURCEFILESFORFIREDETECTION 49 %attrnameisthenameoftheattributetoread.itmustappear %exactlyasinthe.hdffile. %resresult.itmayhavemorethanonedimensionandmaynot %beoftypedouble. %statuscontains0aftersuccessfulexecution,~0otherwise. %changehistory %22/11/2001writtenbykendykutzner f=0;%nofailuresuntilnow %openthefile sd_id=hdfsd('start',filename,'read'); ifsd_id==-1%failure disp(['error:couldnotopenfile"',filename,'"']); f=-1; return; end %searchfordatasetandopenit sds_idx=hdfsd('nametoindex',sd_id,sdsname); sds_id=hdfsd('select',sd_id,sds_idx); ifsds_id==-1%failure outstring=sprintf('error:sds"%s"notfound\n',sdsname); disp(outstring); f=-1; return; end %searchforattributeandreadit attr_id=hdfsd('findattr',sds_id,attrname); [res,status]=hdfsd('readattr',sds_id,attr_id); ifstatus~0 disp(['attribute"'attrname'"notfound']); f=-1; return; end %closedatasetandfile stat1=hdfsd('endaccess',sds_id); stat2=hdfsd('end',sd_id); if(stat1~=0) (stat2~=0) disp(['warning:problemsclosingfile'filename]); end return C.4sdsread.m function[mat,f]=sds_read(filename,sdsname); %SDS_READreadsinScientificDataSetsfrom.HDFfiles %[res,status]=sds_read(filename,sdsname) %filenameisthenameofthe.hdffiletoopen.itisnotparsedin %anyway,soitmaycontainpathinformationanditmustcontain %theextension.hdf %sdsnameisthenameofthedatasettoread.itmustappearexactly %asinthe.hdffile.

CMATLABSOURCEFILESFORFIREDETECTION %resallthedata.itmayhavemorethantwodimensionandmaynot %writtenbykendykutzner %changehistory %25/10/2001writtenfromscratch %statuscontains0aftersuccessfulexecution,~0otherwise. %beoftypedouble. 50 %08/11/2001addedfileclosure f=0; mat=[]; sd_id=hdfsd('start',filename,'read'); ifsd_id==-1%failure disp(['error:couldnotopenfile"',filename,'"']); f=-1; end sds_idx=hdfsd('nametoindex',sd_id,sdsname); ifsds_id==-1%failure sds_id=hdfsd('select',sd_id,sds_idx); disp(outstring); f=-1; return; outstring=sprintf('error:sds"%s"notfound\n',sdsname); return; [dsname,dsndims,dsdims,dstype,dsatts,stat]=hdfsd('getinfo',sds_id); if(stat~=0) ds_start=zeros(1,dsndims);%createsthevector[00]wherewewanttostart ds_stride=[];%wedon'tskipanything ds_edges=dsdims;%andreadtotheend [mat,status]=hdfsd('readdata',sds_id,ds_start,ds_stride,ds_edges); ifstatus~=0 disp(['warning:problemsgettinginfofor'sdsname]); return if(stat1~=0) (stat2~=0) stat2=hdfsd('end',sd_id); %closehdffile stat1=hdfsd('endaccess',sds_id); disp(['warning:problemsreadingsds'sdsname]);

DAdditionalMatlabsourceles DADDITIONALMATLABSOURCEFILES D.1all500m2tiff.m Withthegrowingnumberofavailabledatalestherewereincreasingdicultiestokeeptheoverviewwhichlecontainwhichoverpass.Tosolvethese 51 red,greenandbluein500mresolution,stretchingcontrastineachofthem individuallyandcombiningthemtoargbimage. vertsall500mhdflevel1blesindirectorytotifflesbyreadingbands problemsthesmallfunctionall500m2tiff.mwaswritten.thefunctioncon- needofmemory,abouttwicethesizeofthelargesthdfle. TIFFlesarewrittenishardwiredinthecodeofthefunction. functionall500m2tiff(directory); HDFles.Theparametermusthaveatrailing'/'.Thedirectorywherethe Thefunctionrunsautonomouslyifnoerrorsoccur.Adisadvantageisthe Theonlyinputparameteristhenameofthedirectorycontainingthe %ALL500M2TIFF(directory) %Convertsall500mHDFlevel1bfilesindirectory %ThedirectorywheretheTIFFfilesarewritten %HDFfiles.Theparametermusthavetrailing'/'. %directory:nameofthedirectorycontainingthe %in500mresolution,stretchingcontrastineachof %themindividuallyandcombiningthemtoargb %image. %Caution:needslotofmemory(roughlytwicethesizeof %totifffilesbyreadingbandsred,greenandblue %ishardwiredinthecodeofthefunction. %functionsneeded: %IMPORTHDF %STRETCHthelargestHDFfileindirectory) formatcompact; fori=1:s(1) s=size(f); targetdir='/home/ingest/kendy/tiffs/'; f=dir([directory'*500m.hdf']); %19/11/2001documentedbykendy %changehistory %25/10/2001writtenbykendykutznerfromscratch [nir,red,green,blue]=importhdf([directoryf(i).name]); disp('reading..'); clearnir; disp('stretching..'); disp(['workingon'f(i).name'(size:'num2str(round(f(i).bytes/1024/1024))'m)']); [a,b,c,d]=fileparts(f(i).name);

DADDITIONALMATLABSOURCEFILES rgb(:,:,1)=uint8(round(stretch(double(red))*255)); clearredgreenblue; hdfml('closeall') whosrgb; disp('packing..'); rgb(:,:,3)=uint8(round(stretch(double(blue))*255)); rgb(:,:,2)=uint8(round(stretch(double(green))*255)); 52 disp('writingtiff..'); imwrite(rgb,[targetdirb'.tiff'],'tiff'); disp('clearingup..'); return; res=tmp/ma; res(res>1)=1; ma=.4*max(tmp(:)); tmp=inp-mi; end;pack; functionres=stretch(inp); mi=min(inp(:)); clearrgb; toexportdatafromimapptodimplethematlabfunctionhdf2binwas written.itexportsalldatabandsin1kmresolutiontoabinaryleandsome metainformationfromthegeolocationprocesstoanother.theselescan D.2hdf2bin.m TheDIMPLEsoftware(seesectionA)can'treadHDFlesitself.Asaway pixelcompressionaredone. functionf=hdf2bin(filename,latmin,latmax,longmin,longmax) hdf2bincancuttheimagealonglatitudeand/orlongitudevalues. datatothebinaryle.especiallynocorrectionsforthebowtieeector bereadbydimpleviaitsrawlesimportfacility. %status=hdf2bin(filename,latmin,latmax,longmin,longmax) Thisscriptdoesnodataprocessinginanyway,itjustexportsthelevel1b Tolimittheresultinglesinsize,anareaofinterestcanbeprovidedand %tobereadbydimpleorsimilarprograms %filenameisthebasenameofbothhdffiles.theextensions %.1km.hdfand.geo.hdfareadded. %allremainingparametersgivetheboundarieswherethedata %shouldbecutted.ifnocuttingisdesired,give %ReadsinHDF1km&geofiles,storesoutputinbinaryfiles %-90,90,-180,180asvalues %statusequal0aftersuccessfulexecution,-1afteran %andheight. %erroroccured. %Thefilenameoftheoutputfilesarecreatedwiththe %basename,thentheresolutionoftheimageaftercutting %andlalohe.binforthemetainformationlatitude,longitude %inpixelsandtheextensiondata.binfortheactualdata

DADDITIONALMATLABSOURCEFILES 53 %created02/10/2001bykendykutzner %modified03/10/2001bykendy %-nowoutputaretwofiles,onewithsingleprecision(lat,long&height) %theotherwithuint16(databands) %-bandsarenowinnumericalorder %(12345678910111213lo13hi14lo14hi151617 %18192021222324252627282930313233343536) %-filenamenowwithresolutioninpixels %modified19/11/2001bykendy %-addeddocumentation %someconstants nanreplace=0;%shouldinvaliddatabereplacedbynan? rigidcut=1;%leavedataoutsidespecifiedregionornot? f=0;%nofailureuntilnow %openthegeo-file actname=[filename,'.geo.hdf']; sd_id=hdfsd('start',actname,'read'); ifsd_id==-1%failure disp(['couldnotopenfile"',actname,'"']); f=-1; return; end %readinginlatitude ds_data=readdsbyname(sd_id,'latitude',2,nanreplace); lat=ds_data; %readinginlongitude ds_data=readdsbyname(sd_id,'longitude',2,nanreplace); long=ds_data; %readinginheighinformation ds_data=readdsbyname(sd_id,'height',2,nanreplace); height=ds_data; %finishedwiththe.geofile hdfml('closeall'); %openthe1000m-file actname=[filename,'.1000m.hdf']; sd_id=hdfsd('start',actname,'read'); ifsd_id==-1%failure disp(['couldnotopenfile"',actname,'"']); f=-1; return; end %lat,longandheighthavethesamedimension,so %cuttingthesethree(lat,long,height)together [area,xmin,xmax,ymin,ymax]=coordcut(lat,long,latmin,latmax,longmin,longmax); nlat=lat(ymin:ymax,xmin:xmax); nlong=long(ymin:ymax,xmin:xmax); height=height(ymin:ymax,xmin:xmax); bands=single([]); bands=cat(3,bands,nlat);

DADDITIONALMATLABSOURCEFILES 54 bands=cat(3,bands,nlong); bands=cat(3,bands,height); %readthedataandcutitasearlyaspossible ds_data=readdsbyname(sd_id,'ev_250_aggr1km_refsb',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); ds_data=readdsbyname(sd_id,'ev_500_aggr1km_refsb',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); ds_data=readdsbyname(sd_id,'ev_1km_refsb',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); ds_data=readdsbyname(sd_id,'ev_1km_emissive',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); hdfml('closeall'); %cleanupalittlebit,maybeweneedmem cleartmp; cleards_data; %cutrectangularoralonglat/long? if(rigidcut) areaf=area(ymin:ymax,xmin:xmax); j=size(bands); fori=1:j(3); bands(:,:,i)=single(double(bands(:,:,i)).*double(areaf)); end; end; %nowwearereadytowritethedata: bandssize=size(bands); count=length(bands(:)); if(bandssize(3)~=41)disp('error:notabletoread41bandsofdata');f=-1;return;end; actname=[filename,'.',num2str(bandssize(1)),'x',num2str(bandssize(2)),'.lalohe.bin']; fid=fopen(actname,'w'); iffid==-1 disp('warning:couldnotopenoutputfile"',actname,'"'); f=-1; return; end; %writelat,long,andheightin32bitfloatprecision count=count-fwrite(fid,bands(:,:,1:3),'single'); fclose(fid); actname=[filename,'.',num2str(bandssize(1)),'x',num2str(bandssize(2)),'.data.bin']; fid=fopen(actname,'w'); iffid==-1disp('warning:couldnotopenoutputfile"',actname,'"');f=-1;return;end; %allsuccesivewritesinuint16precision %250mbands1,2 count=count-fwrite(fid,bands(:,:,4:5),'uint16'); %500mbands34567 count=count-fwrite(fid,bands(:,:,6:10),'uint16'); %1kmreflbands8,9,10,11,12,13lo,13hi,14lo,14hi,15,16,17,18,19 count=count-fwrite(fid,bands(:,:,11:24),'uint16'); %1kmemissivebands20,21,22,23,24,25 count=count-fwrite(fid,bands(:,:,26:31),'uint16'); %1kmreflband26 count=count-fwrite(fid,bands(:,:,25),'uint16'); %1kmemissivebands27,28,29,30,31,32,33,34,35,36

DADDITIONALMATLABSOURCEFILES 55 count=count-fwrite(fid,bands(:,:,32:41),'uint16'); fclose(fid); ifcount~=0f=-1; disp(['warning:notalldatawrittencorrectly(',... end;num2str(count),'elementsleft)']); return; %--------------------------------------------------------------------------- %helpfulinternalfunctions %--------------------------------------------------------------------------- %gettingsds_idbyname functionsds_id=getsdsidbyname(sd_id,sdsname) sds_idx=hdfsd('nametoindex',sd_id,sdsname); sds_id=hdfsd('select',sd_id,sds_idx); ifsds_id==-1%failure outstring=sprintf('warning:sds"%s"notfound\n',sdsname); disp(outstring); f=-1; return; end return; %--------------------------------------------------------------------------- %readalldatafromagivensd-set functionds_data=readallsdsdata(sds_id,dsndims,dsdims) ds_start=zeros(1,dsndims);%createsthevector[00] ds_stride=[]; ds_edges=dsdims; [ds_data,status]=hdfsd('readdata',sds_id,ds_start,ds_stride,ds_edges); return; %--------------------------------------------------------------------------- %computethecuttingmatrix function[area,xmin,xmax,ymin,ymax]=... coordcut(lat,long,latmin,latmax,longmin,longmax) iflatmin>=latmaxdisp('warning:latmin>=latmax');end iflongmin>=longmaxdisp('warning:longmit>=longmax');end lat(lat<latmin lat>latmax)=0; lat(lat~=0)=1; long(long<longmin long>longmax)=0; long(long~=0)=1; %areacontainsonly0or1,so %convertareatouint8tosavememory area=uint8(double(lat).*double(long)); yx=size(area); iflength(yx)~=2disp('dimensionerror');return;end %nowlookforboundaries xmin=yx(2); ymin=yx(1); xmax=0; ymax=0; fory=1:yx(1) ifmax(area(y,:))>0 ymin=min(ymin,y); ymax=max(ymax,y);

forx=1:yx(2) DADDITIONALMATLABSOURCEFILES ifmax(area(:,x))>0 endxmin=min(xmin,x); xmax=max(xmax,x); 56 %readanentiredatasetbyname end %--------------------------------------------------------------------------- sds_id=getsdsidbyname(sd_id,dsname); [dsname,dsndims,dsdims,dstype,dsatts,stat]=hdfsd('getinfo',sds_id); functionds_data=readdsbyname(sd_id,dsname,dimension,nanreplace) ifsds_id==-1f=-1;return;end if(dsndims~=dimension)%safetycheck end ifstatus==-1 [attr,status]=hdfsd('readattr',sds_id,attr_idx); ds_data=readallsdsdata(sds_id,dsndims,dsdims); attr_idx=hdfsd('findattr',sds_id,'_fillvalue'); attr=-999;%attributenotfound,usingdefault disp('attribute"_fillvalue"notfoundin"latitude"'); f=-1; disp(['warning:unexpectedformatin"'dsname'"']); return; if(nanreplace)ds_data(ds_data==attr)=nan;end%replacingfillvalueswithnan

EOTHERSOURCECODES EOthersourcecodes andslicer.sectione.3describestheshellscriptwhichcallsallotherprograms. 57 Thissectionlistsotherusedsourcecodes.ThisincludestheprogramsFrameSync TheentireFrameSync.clewouldbefartoolongsoonlythedierences E.1FrameSync.c section3.3. 58d57 <intf_dumpframedecoded=false;/*dumpframesfoundafterremovingpn-randomization*/ totheoriginaloneareshownbelow.thedierencelewascreatedbythe standardgnudiprogram.forexplanationofthechangespleasereferto <intoffbyonebyte=0; <intf_flywheel=false; 60d58 <intn0bits=0; <126,130d120 </*bykendy*/ <intfframed=1; 85,87d82 <puts 160,161d149 <fs_shortstart_sync_match2[16]; 283,307d270 <case'4': <("-4[filename]=dumpframetotheoutputfileafterremovingPN-randomization"); <#else <if((fframed= <{ <++i; <#ifdef_win32 <_S_IREAD _S_IWRITE))<0) if(option_type[i+1]==opt_type_other) f_dumpframedecoded=true; <#endif <{ <0644))<0) sprintf(tbuf,"couldn'topenoutputfile%s", open(option[i],o_rdwr O_CREAT O_TRUNC, open(option[i],o_rdwr O_CREAT O_TRUNC O_BINARY, <} <(void)myerror(1,"-4optionrequiresafilename"); < (void)myerror(1,tbuf); else break; option[i]);

EOTHERSOURCECODES 58 513,514c476 </*if(bitshift>7)*/ <if(bitshift>15) --- >if(bitshift>7) 549,550c511 </*for(i=0;i<8;++i)*/ <for(i=0;i<16;++i) --- >for(i=0;i<8;++i) 555,573d515 </****version2ofmakesyncsearchtable********/ </*bykendykutzner*/ </*I'mworkingonbits16-31ofsyncpattern <becausethesecontainvalidbitsforanypossiblebitshift <ThevaluesarestoredinStart_Sync_Match2*/ <void <Make_Sync_Search_Table2(longpattern) <{ <inti; <for(i=0;i<16;i++) <{ <Start_Sync_Match2[i]=pattern>>i; <} <} <606,607c548 </*for(i=0;i<8;++i)*/ <for(i=0;i<16;++i) --- >for(i=0;i<8;++i) 646,727d586 </*Find_Start_Sync2writtenbyKendyKutzner*/ </*muchfasteronnoisedata,butnotasaccurateastheorginal*/ </*thisdoesn'tmatterbecauseanothercheckisdoneafterwards*/ <int <Find_Start_Sync2(fs_longstartsrch) <{ <inti,j,r; <for(i=startsrch;i<framesize;i++) <{ <r=cvcdu_raw_frame[i+1]; <for(j=0;j<16;j++) <{ <if(r==start_sync_match2[j]) <{ < offset=i; < bitshift=j; < return(1); <} <} <} <return(0); <} <

EOTHERSOURCECODES 749a609,610 >staticintf_flywheel=false; >staticintn0bits=0; 833c694 <if(!find_start_sync2(startsync)) --- >if(!find_start_sync(startsync)) 59 904,911c765 </*bykendy*/ <else 1362d1214 --- >} 914d767 <}/*endfor*/ <weaccept*/ <theslightchancetofindabadsyncmarkerinthedate <{ <Make_Sync_Search_Table2(ShortSync[0]*0x10000+ShortSync[1]); <n0bits=0;/*syncfailed,soresetconsecutivegoodcounter. 1474c1326 <if(vcdu_primary_header.virtual_channel_id== </*****************************/ <{ <write(fframed,(char*)cvcdu_raw_frame,framesize); 1485,1502d1336 </*****IfDumpingdecodedFrames*****/ --- >if(vcdu_primary_header.virtual_channel_id==ch_id_modis) if(f_dumpframedecoded) Ch_ID_MODIS&&VCDU_Primary_Header.SpaceCraft_ID==0x2a) </*****setupthenextframetocheck******/ /*0x2a==TERRA*/ <continue; <} <#endif <break; <lastword=cvcdu_raw_frame[shortframesize]; <CVCDU_Raw_Frame[0]=lastword; 1561,1574d1394 <if(!readdata((char*)&cvcdu_raw_frame[1],framesize)) <{ <#ifdefdebug <fprintf(stderr, <"bogustelemetrypacketsize(nblks=%i,nframesread=%li\n", <Max_Packet_Size) <continue; <#endif <Have_Start_Of_Packet=FALSE; <nblks,nframesread); /*addedbykendykutzner*/ <} if(end_of_prev_packet+start_of_packet> <

EOTHERSOURCECODES 60 1616c1436 < if(imageinfo.fout&&f_flywheel) --- > if(imageinfo.fout) 1728,1730d1547 </*insertedbykendy*/ <Start_Of_Packet+=(Packet_Size+7); </*endinsert*/ 1737c1554 < if(imageinfo.fout&&f_flywheel) --- > if(imageinfo.fout) E.2slicer.c Theprogramslicerisexplainedinsection3.1,basedonctest.cwrittenby RickyLuppinoandshownbelow. #include<errno.h> #include<string.h> #include<stdlib.h> #include<sys/types.h> #include<unistd.h> #include<stdio.h> #include<stdarg.h> #definechunk1024*1024 #defineblk1024 #definefns300 intmain(intargc,char*argv[]) {unsignedcharbuf[chunk]; //unsignedcharres[blk]; charfilename[fns+1]; chartargetfile[fns+1]; chartmpfile[fns+1]; FILE*fp; FILE*of; intfileopen; intstart_chunk; intchunk_count; intcorrel[blk]; intwinval[blk]; inti,j,offset; intk; intcount_array[256]; inthighest; inthighest_v; inthighcor; if(argc<2){ fprintf(stderr,"usage\nslicer<filename>[<targetfile>]\n"); exit(1); }strncpy(filename,argv[1],fns); if(argc>2){ strncpy(targetfile,argv[2],fns); }else strncpy(targetfile,filename,fns);

EOTHERSOURCECODES 61 strncat(filename,".raw",fns); fp=fopen(filename,"rb"); if(fp==null){ fprintf(stderr,"couldnotopen%s\n",filename); exit(1); }snprintf(tmpfile,fns,"%s.slicer%i.tmp",targetfile,getpid()); chunk_count=0; start_chunk=0; fileopen=0; //looparoundanddochunksuntileof while(fread(buf,1,chunk,fp)==chunk){ for(offset=0;offset<blk;offset++){ j=offset; for(k=0;k<256;k++) count_array[k]=0; while(j<chunk){ count_array[buf[j]]++; j+=blk; }//findhighest highest=0; highest_v=count_array[0]; for(k=1;k<256;k++) if(count_array[k]>highest_v){ highest_v=count_array[k]; highest=k; }correl[offset]=highest_v; winval[offset]=highest; }//findmaxcorrelvalueinthechunkandprint highcor=0; for(i=0;i<blk;i++){ if(correl[i]>highcor) highcor=correl[i]; }chunk_count++; //printf("chunk%d=%d\n",chunk_count,highcor); fprintf(stderr,"."); if(highcor==blk){ if(fileopen){ fwrite(buf,1,chunk,of); }else{ of=fopen(tmpfile,"w"); if(of==null){ fprintf(stderr,"couldnotopentempfile%s\n", tmpfile); exit(1); }fileopen=1; fwrite(buf,1,chunk,of); start_chunk=chunk_count; } }else{ if(fileopen){ //closefile fileopen=0;

EOTHERSOURCECODES //printf("attempttorename%sto%s\n",tmpfile,filename); fclose(of); snprintf(filename,fns,"%s-%i-%i.raw",targetfile, printf("%s-%i-%i\n",targetfile,start_chunk,chunk_count-1); if(0!=rename(tmpfile,filename)) start_chunk,chunk_count-1); fprintf(stderr,"error:%s\n",strerror(errno)); 62 }}if(fileopen){ fclose(of); snprintf(filename,fns,"%s-%i-%i.raw",targetfile, //printf("attempttorename%sto%s\n",tmpfile,filename); printf("%s-%i-%i\n",targetfile,start_chunk,chunk_count); if(0!=rename(tmpfile,filename)) start_chunk,chunk_count); } }E.3Theshellscript }fclose(fp); fprintf(stderr,"\n"); return0; fprintf(stderr,"error:%s\n",strerror(errno)); Thissmallshellscripttiesitalltogether.Itcallstheprograms Slicer reformat FrameSync IMAPPconsistingof withalltheirnecessaryarguments.italsocanupdatesomelesneededby Matlabwiththefunctionfiredetection {geolocate {calibrate {L0toL1a IMAPP=$IMAPPDIR/geoimapp.csh #IMAPPDIR=/dd1/IMAPP_RUN rawle. #!/bin/bash IMAPPDIR=/dd2/IMAPP_RUN2 SLICER=/home/ingest/bin/slicer FS2=/home/ingest/bin/fs2 IMAPP.Theonlyparameterittakesisthenameofthe(freshlyrecorded)

EOTHERSOURCECODES MATLAB=/app/matlab12/bin/matlab WGET=/usr/bin/wget REFORMAT=/home/ingest/bin/reformat CP=/bin/cp MV=/bin/mv #weshouldcheckwritepermissioninthecurrentdirectory 63 #checkforcommandlinearguments #maybethenextfourlinesshouldappearin #startoperation $CPutcpole.dat/data/IMAPP/level1a/static $CPleapsec.dat/data/IMAPP/level1a/static #$WGETftp://acdisx.gsfc.nasa.gov/pub/.dbs/ancillary/utcpole.dat #aweeklycronjob $SLICER$1>$1.slicer foractfilein`cat$1.slicer`;do #$WGETftp://acdisx.gsfc.nasa.gov/pub/.dbs/ancillary/leapsec.dat echo"workingonfile$actfile"; #turnthebytesarround #callingstps #thiswouldnotbeneccesarywithfs2butwithstps $FS2-i$actfile.raw-o$actfile.ccsds; #IMAPPcallsthelevel0files.PDS, #initrtheyarecalled.ccsds #insomefuturethefs2linemaybereplacedwithaline $REFORMAT<$actfile.raw>$actfile.tmp #sodoit $IMAPP$actfile.ccsds; #checktheresultsofimapp #echofiredetection\(\'$imappdir/$actfile\'\) $MATLAB-nojvm-nosplash; ls$actfile*-l $MV$actfile.tmp$actfile.raw done

FCongurationFiles FCONFIGURATIONFILES Thissectionlistnecessarycongurationles,mostofthemfortheSTPS suiteofprograms. 64 EDOS_F2Snugget8003100000101 F2Snugget8003320000101 F.1socapps.cfg S2Sstps285000600edos1500160001 CHANNEL0 *END* F.2stps.cfg VERSION3.41 S2Fstps281999320000 STATUS_LOG_FILE_DIR./my ERROR_LOG_FILE_DIR./my CC_MODE1 CC_ENGR_DISPLAY1 DISPLAY_STRINGMY--TelemetryFromFileToFile DSP_PROCESS_UP0 INPUT_BUFFER_SIZE655360 DSP_REC_DIR_NAME./my DSP_RECORD0 DSP_SESSION_TIME_OUT30 DSP_SOCKET_SEND_DATA_HEADER1 DSP_DEVICE0 INPUT_DATA_TYPE0 DSP_SOCKET_SERVER_PORT8002 DSP_SOCKET_OUTPUT_ON1 CHECK_INPUT_BLOCK_CRC0 REC_DATA_MAX_FILE_SZ-1 REC_DATA_TIME_LIMIT-1 DLTP_FRAME_DATA_TO_FILE0 DLTP_UP1 DLTP_PROCESSED_DATA_DIR/home/kendy/stps/my DLTP_SOCKET_HOST_MACHINEingest DLTP_SOCKET_CLIENT_PORT8003

FS_frameLength1024 FS_bitReverseFrames0 DLTP_SOCKET_RECV_DATA_HEADER0 FS_lossOfLockBits0 FCONFIGURATIONFILES 65 FS_autoPolarity0 FS_bitFlipsPerMode03 FS_framesPerMode31 FS_framesPerMode23 FS_framesPerMode19 FS_framesPerMode03 FS_pnDecodeCCSDS0 FS_pnDecodeBeforeReverse0 FS_bitFlipsPerMode13 FS_noFlywheelFramesFlag0 FS_outputFramesState2 FS_bitFlipsPerMode20 FS_bitFlipsPerMode33 FS_bitSlipsPerMode03 FS_bitSlipsPerMode13 FS_bitSlipsPerMode20 FS_bitSlipsPerMode33 FSP1CF FSP2FC FSP31D CHECK_FRAME_CRC0 RSD_ENABLED1 FSP01A FS_fsPatternLength4 RSD_BITS_PER_SYMBOL8 RSD_MAX_CORRECTABLE_ERRS16 RSD_Mo112 RSD_POA11 RSD_VIRTUAL_FILL0 EXTERNAL_MODULES_INPUT_PARAMS_BEGIN RSD_INTERLEAVE4 RSD_MODE1 EDOSdiscardBadFrames1 EDOSoutputServerHostnugget _EDOS_BEGIN_ ESH_VER1 TGT_PORTAA RSDheaderOnly0 EDOSSocOutPort2005

EDOCclcwPortNum0 EDOSinFreq1000000 EDOSoutputCLCW0 EDOSuseUNIXtime0 EDOSoutputSync1 FCONFIGURATIONFILES 66 EDOSclcwMulticastAddress225.2.7.000 TERRAspecificPNDECODE1 EXTERNAL_MODULES_INPUT_PARAMS_END END_PARAMS_ F.3pk.cfg RS_INTERLEAVE=4 #Totalframelengthincludingsync,andRSsymbolsifpresent TOTAL_FRAME_LENGTH=1024 #RSinterleave,setto0ifRSsymbolsarenotpresent #allparametersmusthaveaspaceinthefirstcolumn #allcommentsmusthavea"#"inthefirstcolumn #ConfigurationFileFormat PKT_DIRECTORY=. #ForeachVCtoprocess,enterparameters,nospacesor #commentsareallowedinbetweenparameters. #Basedirectorywherepacketfilesareplaced #1=yes,0=no INSERTZONE_SIZE=0 CLCW_PRESENT=0 (VC2-40havebeencutouttoshortenthisdocument.) VCID=1 CRC_PRESENT=0 VCID=42 INSERTZONE_SIZE=0 CLCW_PRESENT=0 VCID=41 CRC_PRESENT=0

CRC_PRESENT=0 INSERTZONE_SIZE=0 CLCW_PRESENT=0 VCID=43 FCONFIGURATIONFILES 67 (VC44{62havebeencutouttoshortenthisdocument.) CRC_PRESENT=0 INSERTZONE_SIZE=0 CLCW_PRESENT=0 VCID=63

REFERENCES References [1]SteveAckerman,KathleenStrabala,PaulMenzel,RichardFrey,Chris Moeller,LiamGumley,BryanBaum,CrystalSchaaf,andGeorgeRiggs. AlgorithmTheoreticalBasisDocumentDisriminatingClear-Skyfrom 68 [2]RonaldE.AlleyandMaritJentoft-Nilsen.AlgorithmTheoreticalBasis DocumentforBrightnessTemperature.JetPropulsionLaboratory,April CloudwithMODIS.MODISCloudMaskTeam,November1997. [4]M.D.AndrewsandP.E.Ardanuy.Ageneralizedscenarioforclouddetectionusingmodis-n.InGeoscienceandRemoteSensingSymposium, [3]StuartEllisAM.1999-2000AnnualReportoftheSouthAustralian 2000. CountryFireService.SouthAustralianCountryFireService,June 1999. [5]L.L.Bourgeau-Chavez,P.A.Harrell,E.S.Kasischke,andN.H.F.French. [6]S.A.Christopher,MinWang,K.Barbieri,R.M.Welch,andShi-Keng 1990.IGARSS'90,pages1487{1489,1990. ecosystemsusingers-1sarimagery.ingeoscienceandremotesensing Thedetectionandinterpretationofalaskanre-disturbedborealforest [7]SouthAustralianTourismCommission.AboutSouthAustralia,2001. Symposium,1995.IGARSS'95,volume2,pages1246{1248,1995. [8]ConsultativeCommitteeforSpaceDataSystems.SpacecraftID Yang.Satelliteremotesensingofres,smokeandregionalradiative http://www.southaustralia.com.sg/mapway7.htm. energybudgets.ingeoscienceandremotesensing,1997.igarss'97, pages1923{1925vol.4,1997. [10]DundeeSatelliteRecievingStationHomePage.DundeeSatelliteReceivingStationMODISdownlinkpacketformat.http://www.sat.dundee. [9]DepartmentofLandAdministration.SateliteRemoteSensingServices- Firewatch-HotspotDetection,December2001.http://www.rss.dola. wa.gov.au/newsite/noaafd/noaafd.html. ccsds/ccsds.id. List,2001.http://nssdca.gsfc.nasa.gov/anon_dir/active/iacg/ ac.uk/modisformat.html.

REFERENCES [11]CRCforSatelliteSystems.ASTRA.InstituteforTelecommunication [12]LiamGumley,PaulHubanks,andEdMasuoka.MODISAirborneSimulatorLevel-1BDataUserGuide,April1994.http://ltpwww.gsfc. Research,2001.http://www.itr.unisa.edu.au/crcss/astra/astra. 69 [13]DorothyK.Hall,AndrewB.Tait,GeorgeA.Riggs,andVincentV. [14]JunichiKudoh.Forestredetectioninfareastregionofrussiawith nasa.gov/mas/masdug.html. [15]YoramKaufmanandChrisJustice.AlgorithmTheoreticalBasisDocumentMODISFIREPRODUCT.MODISScienceTeam,1998. SensingSymposium,1999.IGARSS'99Proceedings,volume2,pages 858{860,1999. regionofrussiabyusingnoaaavhrrimages.ingeoscienceandremote snow-mappingalgorithm.in1997ieeeinternational,volume:2, pages619{621vol.2,1997.igarss'97.remotesensing-ascientic Salomonson.AlgorithmTheoreticalBasisDocumentfortheMODIS Snow-,LakeIce-andSeaIceMappingAlgorithms,October1998. [16]K.Kawano,J.Kudoh,andS.Makino.Forestredetectioninfareast noaa-15in1998.ingeoscienceandremotesensingsymposium,1999. IGARSS'99Proceedings,volume1,pages182{184,1999. [17]A.G.Klein,D.K.Hall,andG.A.Riggs.Improvingthemodisglobal [19]LockheedMartinMissiles&Space.InterfaceControlDocument [18]ZhanqingLi,A.Khananian,R.H.Fraser,andJ.Cihlar.Automatic IEEETransactionson,39(9):1859{1870,September2001. VisionforSustainableDevelopment. detectionofresmokeusingarticialneuralnetworksandthreshold [20]LockheedMartinMissiles&Space.DirectAccessSystemUser'sGuide (ICD)DataFormatControlBookforEOS-AMSpacecraft(ICD-106). IS20008658C. approachesappliedtoavhrrimagery.geoscienceandremotesensing, [21]J.Minardi,G.B.Marchisio,andR.P.Treder.Spatiallinearmodeling RemoteSensingSymposium,1999.IGARSS'99,pages290{292vol.1, foreos-amspacecraft(icd-107),november1998.is20008696. andforecastingofforestresacrosstheunitedstates.ingeoscienceand

REFERENCES [22]MODISCharacterizationSupportTeam(MCST).ModisGeolocationVersion2ProductFormat,March2001. LatestFilespecs%/MOD03.geolocation.fs.txt. //modis-xl.nascom.nasa.gov/pub/stig_temp/mlinda/www/ ftp: 70 [23]NASA.AquaProjectScience,2001.http://aqua.nasa.gov. [24]NASA.NASA'sVisibleEarth,2001.http://visibleearth.nasa.gov/ [25]NationalCenterforSupercomputerApplicationsattheUniversityof [26]MashNishihama,RobertWolfe,DavidSolomon,FrederickPatt,Jerey Sensors/Terra/MODIS.html. Illinois.HDFUser'sGuideVersion4.1r4,December2000.ftp://ftp. ncsa.uiuc.edu/hdf/hdf/documentation/hdf4.1r4/users_guide. [28]RobertE.Wolfe,DavidP.Roy,andEricVermote.Modislanddatastorage,gridding,andcompositingmethodology:Level2grid.Geoscience Location:AlgorithmTheoreticalBasisDocumentVersion3.0.MODIS [27]ArnoPeters.DieneueKartographie/Thenewcartography.Friendship Press,1983. ScienceDataSupportTeam,August1997. Blanchette,AlbertFleig,andEdwardMasuoka.MODISLevel1AEarth [29]YilinZhao.Vehiclelocationandnavigationsystems.ArtechHouse, andremotesensing,ieeetransactionson,volume:36issue:4, 36(4):1324{1339,July1998. 1997.AppendixB.