Adelaide,December7,2001 KendyKutzner

Transcription

1 Telecommunications Institutefor Research forfiredetectioninaustralia ProcessingMODISData Prof.WilliamCowley,ITR KendyKutzner MichaelSloman,ITR Supervisors: InformationEngineering SchoolofElectricaland December7,2001

2 publishedorwrittenbyanotherauthorexceptwhereduereferenceismade inthetext. Ideclarethatthisworkdoesnotincorporateanymaterialpreviously2 Adelaide,December7,2001 KendyKutzner Australia,inordertofullltherequirementsfor Studienarbeittoreachthedegreeof'DiplomingenieurElektrotechnik' ThisworkissubmittedtotheInstituteforTelecommunicationsResearch, Thehonorspartoftheworkisespeciallypresentinsection4. Finalyearproject(hons)attheschoolofElectricalandInformation atthechemnitzuniversityoftechnology,germany Engineering,UniversityofSouthAustralia

3 ListofFigures TableofContents CONTENTS 43 2Introduction ListofTables 1Abstract 2.3TerraandMODIS ConventionsandAbbreviations ITRandASTRA SouthAustraliaandBushFires OverviewoverDataow UsedFileFormats Rawrecordeddata...9 3DataProcessingtoLevel1b 2.7BowTieEectandMapProjections ProductionDataSets HierarchicalDataFormat IntroductionintoFrameSynchronization Preprocessing FileSizes OtherusedFileFormats TheProgramFrameSync FrameSynchronizationwithSTPS BitSynchronization FrameFormat FireDetection 3.4.3stpsccanddltp reformat AlgorithmsforPreparingFireDetection IMAPP f2s CreationandEvaluationoftheSnowMask pk CloudDetection EvaluationoftheLand/SeaMask...24

4 CONTENTS 4.2FireDetectionAlgorithms AlgorithmsusingtheResultsofFireDetection ImplementationofFireDetection MapProjection Smoothing Recommendation 5Conclusions 34 CMatlabSourceFilesforFireDetection BGlossary ADimple C.1firedetection.m C.2radiance2teff.m...47 EOthersourcecodes DAdditionalMatlabsourceles D.2hdf2bin.m...52 D.1all500m2tiff.m...51 C.4sdsread.m...49 C.3attributeread.m...48 E.1FrameSync.c...57 E.2slicer.c FCongurationFiles References E.3Theshellscript...62 F.2stps.cfg...64 F.3pk.cfg...66 F.1socapps.cfg

5 LISTOFFIGURES ListofFigures 1FlowofdatafromASTRAtoredetectionresult...9 2BowTieEect DownlinkPacketFormat ListofTables 4DataowinSTPS Sampleoutputfromthescriptfiredetection Someimportantsettingsinthecongurationleofstpscc ExamplesofFileSizes Pictureofdetectedres Possiblevaluesforland/seamask Listofdetectedres...36

6 mentoftheterrasatellitetodetectbushresinaustralia.thisincluded preprocessingthedemodulatoroutput,bitsynchronizationandreassembly 1ABSTRACT 1Abstract TheaimofthisworkwastouseremotesensingdatafromtheMODISinstru- 6 ofdatapackets.imappwasusedtodothegeolocationanddatacalibration. similar. ferencetestsandbackgroundcomparisons.theresultswereprojectedina bynasaandredetectionsbasedonothersensorsandfoundtobevery rectangularlatitude/longitudemaptoremedythebowtieeect.algorithms intheavailabledata.theresultswerecomparedwithredetectionsdone wereimplementedincandmatlab.itprovedtobepossibletodetectres Theredetectionusedacombinationofxedthresholdtechniqueswithdif-

7 Itbeginswithageneralintroductionwhyremotesensingisusefulforredetection.Thentheusedconventionsandleformatsareexplainedandatthe 2INTRODUCTION 7 2Introduction Thissectionshouldgiveanoverviewoverproblemsthisworkisdealingwith. continent.itslargeagricultural,shrubandforestedareasposingahighre SouthAustraliaisthethirdlargeststateinAustraliawithanareaofmore 2.1SouthAustraliaandBushFires endsomeoftheproblemsaredescribedindetail. dicultbecauseofthevastareaandsparsepopulationinmostofthestate. than9:841011m2[7].itisoftenreferredasthedrieststateinthedriest supporttheearlyredetection. hazardinmostsummers.therewere3204hectaresburntandover5million 2.2ITRandASTRA AustralianDollarslostinruralresinthe1999/2000summeralone[3]. Thisworktriestodeterminethepossibilityofusingremotesensingdatato Thisshouldmakecleartheneedforfastredetection.Firereportingis TheInstituteforCommunicationsResearch(ITR)hasgottheASTRAsince ableparabolicdishandiscurrentlyconguredforxband.itispossibleto December1984[11].ASTRAstandsforAutomaticSatelliteTrackingResearchAntennaandisoriginallyfromNASA.Itisa6.8mhydraulicalsteer- SATandRADARSAT[11].Inthisworktheinterestliesonthesatellites receiveearthresourcesatellitessuchasers,jers,spot,mos,land- Terra(formerlyEOS-AM)andAqua(EOS-PM). TheTerrasatelliteispartofNASA'sEarthObservingSystem(EOS).Itwas 2.3TerraandMODIS 150megabitperseconddevelopedatITR. withersdem2,acongurabledemodulatorcapableofdatastreamsupto AfterreceptionwithASTRAthesignalgetsimmediatelydemodulated launchedindecember1999andisreferredastheagshipoftheeos.since themoderateresolutionimagingspectroradiometerormodis.thissensor seestheentireearthsurfaceeverytwodaysandacquiresdatain36dierent February2000itisfullyoperational.Onofthemaininstrumentitcarriesis bands.

8 alreadygoodcoveragewillbeevenbetter.untilnow,everydayoneoverpass ment.aquawillbelaunched,accordingto[23],inmarch2002.thenthe phenomenology.theeos-pmoraquaplatformwillcarrythesameinstru- 2INTRODUCTION ofitschannelsareespeciallydesignedforterrestrial,atmosphericorocean MODISismeanttobeofuseforaverybroadrangeofscientists.Many8 castfacilityorcanbestoredandtransferedtotheearthlaterindirectplay- backmode.itrhasthecapabilitytoreceivethecontinuousdirectbroadcast datastreamfromthemodisinstrument. canberecordedinthemorning,withaquatherewillbeoneintheafternoon (That'swhythesatellitewascalled-PM). DatacollectedbyMODISistransmittedimmediatlyviathedirectbroad- ForalistofabbreviationspleaserefertoappendixB. 2.4ConventionsandAbbreviations maybeprovideusefulresultstoorganizationslikethecountryfireservice. Thegoalwastoautomaticallydetectbushresfromreceiveddata. Thisworkisthetrialtoprocessthedataabovethelevel1bstageand puterunlessotherwisestated. Themostoftenusedsymbolsinclude: 'CommandLine'usuallyreferstoashshellonaSunOSorLinuxcom- BRadianceatwavelength TBrightnesstemperatureconvertedfromB Reectanceatwavelength Centerwavelengthofachannel(inm) Latitudevalues dentofeachotherdierentlevelshavebeendenedandwidelyaccepted. 2.5OverviewoverDataow havetobeprocessedafewsteps.tomaketheseprocessingstagesindepen- Firedetectiondoesnotworkimmediatelywiththereceiveddata.Theles Longitudevalues Level0dataisdenedasthereconstructedCCSDSpackets(seesection3.2.2) 2.6.3)producedbyIMAPP(seesection3.5).Level1bdataistheradiometricallycalibrateddata.Furthermoregeolocationinformationisavailablein removed.level1aisanintermediateproductbasedonhdf(seesection thelevel1bproduct.thisincludesancillarydatalikethepositionofthe Rawdataisjustrecordedfromthesatelliteandnotprocessedinanyway. withallcommunicationartifactsincludingduplicateanderroneouspackets spacecraftorsensorandsolarangles.

9 Thisworkstartsatthepreprocessingstageswhenrawlesaretouchedfor anddemodulatedwithersdem2.thesestepsarenotpartofthisreport. detection. 2INTRODUCTION Inlevel1bproductsenoughinformationisavailabletodoasuccessfulre Theowofdatacanbeseeningure1.RawdataisreceivedbyASTRA9 clusionofobviouslybaddataandthebitreversion.level0dataisproduced thersttimeintheconversiontolevel0data.preprocessingincludestheex- byframesyncorstps(seesections3.3and3.4).imappisresponsiblefor stepstolevel1b.theactualredetectionisbasedonthisoutputandis doneinmatlab. Figure1:FlowofdatafromASTRAtoredetectionresult 2.6UsedFileFormats 1 0 slicer & reformat 2.6.1Rawrecordeddata raw AfterreceptionofthedatabyASTRAanddemodulationbyERSDEM2 FrameSync STPS ASTRA dataisrecordedtocomputerles.theseleshavenoparticularformatand level 0 IMAPP level 1 broadcastmodeandaaverageoverpassdurationlike12minutesthetypical DuetotheTerradatarateof13.125megabitpersecondintheuseddirect containaconcatenationofbits.usuallytheselescarrytheextensionraw. Dimple detected fires lesizeisaroundonegigabyte.amoredetaileddescriptionofthedataand Matlab (picture and list) consecutiveccsdspackets(seesection3.2.2).thesepacketstreamshould problemsassociatedwithitareinsection ProductionDataSets Datainlevel0isoftencalledproductiondatasets.Thesedatasetsconsistsof

10 notcontainanyartifactsfromthecommunicationanymore.allerroneous anddoubledpacketsshouldberemoved. 2INTRODUCTION ditionalheaderscontainingmetainformation(edosheaders),butinthis projectimapp(seesection3.5)isusedforlevel1processinganddoesn't SomeprogramslikefurtherprocessingsoftwarefromNASArequiresad HierarchicalDataFormat.ccsdsbecausetheycontainCCSDSpackets. smallerthanrawdatales(seesection2.6.5). needsuchheaders. Theextensionoftheselesisusually.pdsforproductiondatasetor Sincetheselesdon'tcontaintheprotocoloverheadtheyarealittlebit umentedandsupportedandmainlyusedforscienticdataexchangeand forsupercomputingapplications.itisaopenleformat,itiswelldoc- archival. TheHierarchicalDataFormat(HDF)wasdevelopedbytheNationalCenter oneishdfversionve.inthisprojecthoweveronlyversionfourisused, format. itisstillfullysupportedbyncsaandmanyavailabletoolsoperateonthis Therearetwoentirelydierentandincompatibleformats.Thenewer bandsarestoredtogetherinonedataset. sets.herethemaximumdimensionusedisthree,sincemanytwodimensional scriptionofdatasetsorstorageofadditionalscalingvalues. attributeswiththedata.thisisusedforexampleforhumanreadablede- OneofthemainfeaturesofHDFisitssupporttostoremetadataor theprovidedinterface.allaccessestothedataaredonethroughalibrary andnolowlevelknowledgeoftheleformatisnecessary. AnadvantageofHDFoverotherleformatsisthestrictneedtoused Anotherinterestingfeatureistheabilitytostoremultidimensionaldata 2.6.4OtherusedFileFormats Partsofthisprojectrequirethestoreofimagedata.Forthispurposethe intheunixwayforlineendingswith0x10asalineendmarker.toconvert tionlesmentionedandallsourcelesformatlabandc.textlesarestored andstores24bitcolorimages. formattiff(taggedimagefileformat)waschosen.thishasseveral platformsandsoftwarepackages.itiscapableoflosslessdatacompression reasonsincludingthewidesupportforthisformatacrossdierentcomputer Textlesincludetheoutputoftheredetectionalgorithm,thecongura-

11 resonecantype thatfurtherprocessingiseasy.forexampletogetthenumberofdetected themtoanydos/windowsbasedplatformthetoolunix2dos<filename> canbeused. 2INTRODUCTION Thetextleproducedbytheredetectionscriptareformatedinaway 11 wellasthepositionoftherepixelinthenalmap. grep-v^#lenamejwc-l 2.6.5FileSizes lineisprinted.itcontainsthelatitudeandlongitudevaluesofthepixelas outputlenamesarestated.aftertheheaderforeachrepixelfoundone atthecommandlineprompt.theleheadercontainsinformationabout theprocessingdoneandtheproportionsoftheproducedimage.inputand withtheselesmustbeawareofthis. getanideaoflesizes,seetable1.allprogramsandalgorithmsdealing Remotesensingdatatendstobelarge,especiallywithhigherresolution.To recordingdate29/11/200105/12/2001 afterframesyncb330,155,562250,984,764 level1a geolocationdata50,765,78338,511, mdata initiallesizea411,041,792312,475, ,492,136218,165,563 98,713,60477,012,996 Table1:ExamplesofFileSizes 500mdata 250mdata resulting.tile8,072,2863,074,088 arawdata blevel0 230,296,084174,759, ,500,060181,742,949 Tieeect.Itwasrecordedon30/08/2001andshowstheEerieandYork andtoadistortedimageattheedges.figure2isanexampleofthebow TheMODISinstrumenthasthenominalspatialresolutiononlynearthe thandoubleofthenominalone[26].thisleadstooverlappingofswaths nadirline.asaresultofthemotionofthescanningmirrorintheinstrument, dataelementsattheedgeofthe110degreeswathwidthhaveasizeofmore 2.7BowTieEectandMapProjections

12 PeninsulainSouthAustraliainthe500muncalibratedcolor.Notethewrong aspectratioasaresultofnonsquarepictureelementsfrommodis.another unpleasantartifactisthattheimageorientationisnotnorthbutintheight pathofthesatellite. 2INTRODUCTION IMAPPprovidesthegeolocationdataforevery1kmpixel.ThisinformationcanbeusedtocorrectbothBowTieanddistortioneects. TransverseMercator)andrectangularlatitude/longitude.Goodexplanation dierentapproachesareknown.twocommononesaretheutm(universal Rectangularlatitude/longitudemapsthesphericalsurfacetoarectangle Earth'snearsphericalsurfaceisnoteasilyprojectedtoaplane.Many 12 canbefoundin[27].forconversionseeforexample[29].therstprovides muchbetterresultsforsmallerareasbutprovednotpracticalforrecorded imageswithsizesupto3420x2332km. disadvantageofdistortionnearthepoles(apointismappedtoaline)but withlatitudeandlongitudeasxandycoordinates.thishastheapparent thisisnotahugeprobleminthelandareaofaustraliawithlatitudevalues lessthan40degrees.figure2:bowtieeect

13 3DATAPROCESSINGTOLEVEL1B 3.1Preprocessing Afterrecordingtherawdatafromthesatellitesdirectbroadcastfacilitya 3DataProcessingtoLevel1b 13 numberofprocessingstepsarenecessary.therstisthepreprocessingto getridoflargeramountsofbaddatarecordedfromthedemodulator.there Itperformsakindofanautocorrelationoftherawdata.Ifthiscorrelationpeaksat1024becauseoftherepeatedsyncmarkers(section3.2.2),the programassumesgooddataandpassesthedatatothenextsteps. Thisisdonewiththesmallprogramslicer,whichisbasedonasimilar forsomereasonorwhenthesatellite'stransmitterisswitchedoforthe programfromrickyluppino.sourcecodecanbeseeninappendixe.2. receptionofdeepspaceprobes. Furtherbaddatablocksmaybetherewhenthetrackingofthesatellitefails areblocksofunusabledataatthebeginningandtheendofeachrecording. availableinitr.theprogramreformat(seesection3.4.1)cancelsthisreversion. 3.2IntroductionintoFrameSynchronization scribedinsection3.4.thisprogramcan'tdealwithbytereverseddata AnotherpreprocessingisnecessaryfortheuseoftheSTPSpackagede- issaved.anothernameforthistypeofdataisrawtelemetrychanneldata. Asdescribedbefore,theoutputofpreviousstagesistheconcatenationof thelestherawrecordingfromallbroadcastcommunicationofthesatellite havetobeunderstoodandthedataweareinterestedincanbeextracted. bits.untilnow,noalignmentisdoneandnoneofthebitshasameaning.in 3.2.1BitSynchronization Units(CADU)orframes.AllCADUshavethesamelengthof1024octets. DatainthesatellitedownlinkisorganizedinsocalledChannelAccessData Thenexttaskistobytealignthesebits.Afterthis,theprotocolsused Tosupportthesearchforframestarts,therstpartofeachframeisan AttachedSyncMarker(ASM)of32bit.Thechosenbitpatternforthe channelsbyconsultativecommitteeforspacedatasystems(ccsds).it Terrasatelliteis orinhexadecimal notation0x1acffc1d.thissyncmarkerisrecommendedfortelemetrydata willbeusedbyothersatelliteslikeaqua.bitsynchronizationisconsidered

14 3DATAPROCESSINGTOLEVEL1B donelike donewhenthesyncsequenceisrecognized.however,furthercheckscanbe Arethereothersyncmarkers1024octetsbeforeandafterthisposition? 14 demodulatorcongurationsthepolarityofbitsmaybeinverted.theknown patterncanbeusedtodetectthisconditionandswitchallthebitsback. WithERSDEM2andTerrathisproblemdoesnotoccur. Doesthecontentoftheframemakessensetothenextprotocollayers? Doesthecontentoftheframepassesanyerrortest? 3.2.2FrameFormat Anothertaskcanutilizetheattachedsyncmarkerpattern.Insome ThenextstepistheextractionofMODISpackets,sincethedatastreamuntil nowmaycontaindatafromotheron-boardinstrumentsorothertelemetry moved. data.moreovertheprotocoloverheadforthetransportlayerhastobere- calledchannelaccessdataunit(cadu).eachcaduhastheconstant sizeof1024byte.therst32bitsofeachcaduistheattachedsync marker.thesecondpartisexactlyonecodedvirtualchanneldataunit (CVCDU)of1020bytes. Asmentionedbefore,datafromthespacecraftisorganizedinframes satellitedownlink). and128bytesappendedreedsolomonforwarderrorcorrectioncode.the datachannels(forexampledierentinstruments)overonephysicallink(the purposeofvirtualchannelsisthepossibilitytotransmitseveraldierent EveryCVCDUconsistsof892bytesVirtualChannelDataUnit(VCDU) CCSDSpackets. spacecraftidassignedbyccsds.terrahastheid0x2a,aquawillhave 1intheTerraandAquasatellites.Thenexteldstatesthegloballyunique theid0x9a[8]. intothevcduheaderandampduheader.thesecondpartconsistsof TheVCDUheadercontainstheversionnumberoftheprotocolwhichis Thevirtualchanneldataunithastwomainparts.Therstpartdivided thesamechannelnumberformonevirtualchannel.fortheassignmentof foremptyvcdus. channelnumberssee[19]page61.aspecialchannelidis0x3fwhichisused channel.allvirtualchanneldataunitswiththesamespacecraftidand Thespacecraftidenticationeldisfollowedbythenumberofthevirtual

15 3DATAPROCESSINGTOLEVEL1B inthetelemetrydatastream. channel.withthiselditispossibletodetectduplicatedormissingparts Thenexteldisacountofvirtualchanneldataunitsforeachvirtual 15 Figure3:DownlinkPacketFormat end of previous CADU CADU (1024 octets) start of next CADU Attached Sync Marker (4 octets) Coded VCDU (1020 octets) canhavedierentlengths(forexampleday,nightorllpackets),eachpacket VCDU (892 octets) Reed Solomon Code (128 octets) header (6 octets) data (886 octets) thetaskoftherawtolevel0softwaretoreassemblethesesfragments. CCSDSpacketintheMPDUheader.ItispossibleforaCCSDSpacketto besplitovermultiplevcdus.eventheheadereldofaccsdspacketmay carriesitsownlengtheldandthereisapointertothebeginningoftherst bedivided.itisalsopossibletohavemorethanonepacketpervcdu.itis AftertheMPDUheadertheCCSDSpacketsfollow.Sincethesepackets M_PDU header (2 octets) end last packet(s) (variable size) start next transitiondensityinthedownlinkstream,bitsarexor-edwithapseudo randomnoisesequenceh(x)=x8+x7+x5+x3+x1 Toavoidrepeatingpatternsbesidesthesyncmarkerandensuredata Primary Header Secondary Header MODIS header MODIS data Checksum errorcorrectiontoformacvcdu. [20].ThissequenceisinitializedtoallOne'satthestartofeveryVCDU. InordertosuccessfullydecodeframesthestepsareforTerra ThesequenceofthetwodierentcodingsvariesbetweenTerraandAqua. EachVCDUisfollowedby128octetsReedSolomoncodeforforward

16 3DATAPROCESSINGTOLEVEL1B whileforaquathestepsare 2.PNDecodeVCDUbutnottheVCDUheader 1.DecodeReedSolomon(RS)forentireVCDU(CADUwithoutsync markerandrscode) TheProgramFrameSync Toperformthetasksdescribedabove,theprogramFrameSyncwaswritten attheuniversityofhawaii.theoriginalversiondidnotworkverygoodon 1.PNdecodeCVCDU(includingVCDUheaderandRScode) thedataavailableatitrsoafewchangesweredone. 2.DecodeRSforentireVCDU. with16bitdata,buttheorginalversioncheckedonlyforeightdierentbit ternappearsatthebeginningofablock.thechangedvariantshiftsthe syncmarkerthroughallpossibilitiesandcomparestheorginaldatawiththe shiftedsyncpatterns,whichismuchfaster.internallytheprogramworks originalversiondidthisbybitshiftingthewholedatauntilthemarkerpat- shifts.obviously16dierentbyteshiftsarepossiblewithwordsizedata. Atrst,theprogramstartstolookforthesyncmarkerpattern.The Thesearchfortheremainingeightcontingencieswasadded. CCSDSpackets.Theseincludechecksforpacketsizeandcorrectnessof subjoined. headerelds.tosupportthedebugingafacilityfordumpingvcduswas Alsoaddedwereanumberofcheckstopreventtheoutputofinvalid 3.4FrameSynchronizationwithSTPS andadierentspacecraftidwillbeused. satellitewillbelaunchedbecausethedecodingsequenceisslightlychanged suchdatawasnotavailable.furtherchangesmustbedonewhentheaqua TheprogramFrameSyncwasnottestedwithMODISnightpacketssince Asdescribedbefore,theprogramFrameSyncasseveraldisadvantages.Althoughtherehavebeensomeimprovements,afewbugsremain.Forexample, (Theformatofthedatastreamisexplainedin[10].)IfaCCSDSpacket syncmarkeratthebeginningofit.theprogramignoresthereedsolomon codesprovidedbythesatellite.anotherwayofcheckingframeswouldbe lookingforthenextsyncmarker,butframesyncdoesn'tdothis. anydamagedframeisstillregardedascompleteaslongthereisaattached Atthenextstage,CCSDSpacketshavetobeassembledfromtheVCDUs.

17 3DATAPROCESSINGTOLEVEL1B Thenextarrivingfragmentofapacketisappendedtothersthalf,ifthe isnotnishedinthecurrentframe,framesyncwaitsforthesecondhalf. VCDUcounteristhesuccessorofthepreviousone.Thisworksne,because above,thedatausedtocompletepacketsisnotchecked.forexample,ifthe packagesfromallothervirtualchannelsarediscardedearlier.asmentioned 17 packetwouldresult.alengthcheckisdone,butmodisnightpacketswill pointertotherstccsdspacketinthesecondframeiscorrupt,ainvalid (GSFC)provedtobeabletosolvesomeoftheproblems.STPSstandsfor SatelliteTelemetryProcessingSystem. theendofeachmodispacket. failthischeck. inthepackagestpsv3.41.tar.gzfromnasagoddardspaceflightcenter Theoveralltaskisnotdonebyasingleprogram.InNASA'sapproach That'swhyanotherwayofdoingthistaskwasguredout.Thesoftware Finally,thepacketsarewrittenwithoutcheckingtheavailableCRCat shouldbe reformatisnecessary. theprogramsf2s,stpscc,dltpandpkareinvolved.additionallythetool 'exportstpslocation=/home/.../stps'. variableintheenvironmentpointstothebasedirectoryofstps.inthecsh ortcshshelltheneededcommandlookslike 'setenvstpslocation/home/.../stps'.withtheshorbashshellsit Forallaboveprogramsexceptreformatandf2sitisnecessarythata onepage reformat Itwasfoundthatthe.rawlesavailableinITRarebyte-reversed.Thisis notaproblemfortheprogramframesyncbecausetheswitch-rismade Theinteractionbetweenthedierentprogramscanbeseeningure4 forthiscondition.howevertheprogramdltpcannotdealwithsuchlesso whichcandothistask.ifthebitsinthelesareordered ,afterrunningthroughthetoolthesequenceischangedto thehavetobeturnedexternally.michaelslomanwrotethetoolreformat reformat<inputle>outputle entirelesbytebybyte,itdoesnotmatterwhethertheexecutingcomputer islittleendianorbigendianmachine. Theprogramactsasalter,soacommonwaytocallitis Sincetheinputlesarenotbyteorwordswappedandreformatreadsthe

18 3DATAPROCESSINGTOLEVEL1B 18 Figure4:DataowinSTPS bit reversed.raw file f2s on ingest TCP port 8003 stpscc on nugget dltp on nugget TCP port 2005 pk vc42ap64.dat on nugget

19 3DATAPROCESSINGTOLEVEL1B 3.4.2f2s Thetoolf2sreadsalefromdisk,waitsforaconnectionattemptona speciedtcpportandahitfromthekeyboard.aftertransmittingthele viathisconnection,theprogramexits. 19 dltpandpkcanonlyrunonbigendiancomputers,sothedatahastobe transferedtheresomehow.secondtheprogramdltpdoesn'treadfromdisk les,onlyfromtcpports. ofsuchaleisincludedinsectionf.1.forf2sitisnecessarythatthele containsalinestartingwithf2sandonestartingwith*end*.anylinenot Therearetworeasonsforusingthistool.Firstthenextstageprograms startingwithakeywordisregardedascomment. arenameoftheremotehost,wheredltpisrunning.itcanbeanipaddress. Theprogram'sexecutioniscontrolledbyacongurationle.Anexample TCPportnumberfortheconnection.8003isrecommended. Blocksizefortransmission.Smallvaluesincreasetransmissiontime. AfterthekeywordF2Stherehavetobesevenparameters.Theirmeaning Numberofrepeats.Inthisapplicationitisnotusefultosetittoother Waittimebetweenpacketsinmilliseconds.Thisvaluecanbeusedto valuesthan1. reducesystemloadonthetargetcomputer. more islargeenoughsolargernumbersdon'tspeeduptheprocessany transmitadditionaledos-headerinformationinthetcpstream.since Modeofoperation.Avalueof0meansopeningthesocketinserver dltpgetsconfusedwiththisinformation,itsuseisnotrecommended.other Ifthelastvalueissetto1insteadof0,f2soutputsafewmorestatus Thereareslightlydierentvariantsoff2s.Thevariantscallededosf2s mode,1clientmode.hereonlyservermodeisused. dierencesarewhetherf2sreadsintheentireleatstartuporreadsitin inblocksasnecessary.theauthoroff2sclaimstherstmethodwouldbe messages,whichcanbeuseful. memory,thatswhythesecondmethodhasbeenused. faster.thiscouldnotbereproduced.thelesarelargecomparedtosystem

20 3DATAPROCESSINGTOLEVEL1B 3.4.3stpsccanddltp istheircloseinteraction.thechannelcontrollerstpsccisresponsiblefor Thereasonwhythesetwoprogramsaredescribedtogetherinonesection readinginthecongurationle,startingandstoppingthechildprocessdltp 20 andprintingoutprogressinformation. nicatingviaudpwithagraphicaluserinterface.thesefeatureshavenot acongurationle.onthecommandlinetherehavetobefourparameters. beenusedhere. IthassomeothercapabilitieslikereadingfromDSPdevicesorcommu- itaterminatingsignaleitherviacontrol-corviathekillcommand.ifstpscc arenecessaryforasuccessfulstart,atinyshellscriptrunstpsisprovided. nicationwiththeguiandarenotimportanthere.becauseallparameters Therstonenamesthecongurationle,alltheothersdealwithcommu- Asusual,stpsccisconguredviaparametersonthecommandlineandvia receivessuchasignal,itshutsallopenconnectionsdownandkillsdltp.then theprogramexits. Oncestarted,stpsccrunsforever.Thewaytoendtheprogramistosend stepisthereedsolomondecoding.anumberofbiterrorscanbecorrected. modules,wherepnderandomizationisdone.intheoutputmodulethenext Iftheremoreerrorsthancanberecovered,theframeisdiscarded. thelockstateisentered.thenframesareforwardedtotheedosoutput fromf2s.afterthesyncpatternisfoundseveraltimesattheexpectedplace AftertheframewassuccessfulPNandRSdecoded,itsendviaanother Theimportantprogramhereisdltp.ItreceivesviaaTCPlinkthedata TCPlinktopk.InthisdatastreaminfrontofeachframeaEDOSheader workinglealllineshavetobeincluded,eveniftheyhavenomeaningin issend,whichisnecessaryforpktounderstandthem. thiscase,likethelinesaboutthedspprocess. thelearenotallowed. lineshavetoappearandthesequencemustnotbechanged.commentsin F.2.Thewaystpsccreadsinthisleisnotespeciallysmart,thatswhyall Intable2theexplanationofsomeimportantlinescanbeseen.Ina Anexampleofaworkingcongurationlehasbeenincludedinsection OnLinuxcomputersitcanbecompiledonlywithchangingtheMakelesand somesourcecodelines.itdoesn'trunwellonlinuxmachines. ThissoftwarewasdevelopedonSunmachinesusingtheGNUCcompiler.

21 3DATAPROCESSINGTOLEVEL1B 21 errorlogledir.. VERSION3.41 Table2:Someimportantsettingsinthecongurationleofstpscc Necessarybecausetheformatofthiscongurationlechangedbetweenversions inputdatatype0 inputbuersize statuslogledir.. dspprocessup0 Indicatesdataisnotformatedinanyway.If Herenoattemptismadetoreaddirectlyfrom Adirectorywheresomeprogressinformation Amountofdatareadinatonce DSPs abletoreaddata. thissettingisnotcorrect,dltpwouldnotbe arestored. dltpframedatatole0becausedataissendtothenextstageitisnot checkinputblockcrc0framesarenotprotectedbyacrc. dltpup1 Thislinebringsupthedltpprocess. fsoutputframesstate2meaningofstateormode:0-search,1-check, fspndecodeccsds0 dltpsockethostmachinewheref2sisrunning fsbitreverseframes0 PNdecodingisdonelaterintheEDOSmodule,whichisconguredattheendofthele. necessarytosaveit. notthesameasreformat EdosDiscardBadFrames1 checkframecrc0 rsdheaderonly0 EdosOutputServerHostThecomputerpkisrunningon rsdenabled1 fsframespermode19 Framesdon'thaveCRC's 2-Lock,3-Flywheela. EdosOutputSync1 TerraSpecicPNdecode1 aflywheelmodeindltpisnotcomparabletoywheelmodeinframesync

22 3DATAPROCESSINGTOLEVEL1B 3.4.4pk ThepacketprocessorpkisusedtoextractCCSDSdatapacketsfromthe streamofframes,reassemblethemandseparatethemfordierentvirtual channelandapplicationid.herevirtualchannel42andapplication64 22 lesarenamedvc<vcid>ap<appid>.datsoheretheoutputlecalled thecongurationleisgiveninsectionf.3. file>set.ifnolenameisgiven,thedefaultispk.config.anexampleof startedfromthecommandline,usuallywiththeswitch-s<configuration (MODIS)inthischannelareofinterest. vc42ap64.datistheinterestingone. ForeachchannelandeachapplicationIDaseparateleiswritten.All TheprogramresidesinthesubdirectorypkSERVICEPROCESSOR.It thenextstageofdataprocessinganddoesn'tunderstandthem.that'swhy pkwaspatchedtooutputonlyccsdspackets.theoutputlecanthen bemovedtothecomputerrunningimapp.untilnowthisisdoneviaftp, latermaybeitshouldbedonevianfs. onthisfeature.itdoesn'trunonlittleendiancomputers. NormallypkwritesEDOSheadersintheoutputles.IMAPPisusedin fromtheoperationalmodisprocessingsoftwaredevelopedatnasagoddardspaceflightcenter.itconsistofthreemaincomponents. 3.5IMAPP IMAPPstandforInternationalMODISProcessingPackageandisderived Theprogramwasimplementedonabigendianmachineandisdependent model(dem)isnecessarytoprovidealtitudeinformation.theprogram thetimeorderingofthepackets. geolocatecanalsodothegeolocationwithoutcorrectionfortheterrain CCSDSpackets.ThentheygetreformattedintoanHDFlebytheprogram L0toL1a.Thisprogramdoesafewchecksonthedata,forexampleitveries Thesecondstepisthegeolocationofthedata.Hereadigitalelevation Therstpartconvertslevel0datatolevel1abyexaminingthecontentof ampleofdataprocessingcanbedimpleforclassication(seeappendixa). correctioniskepton,becausetheoutputismoreaccurate. whichcansavealittlebitofprocessingtime.thenthedemisnotnecessary, asavingofharddiskspaceofaboutthreegigabytes.inmostcasestheterrain theproductionoftheoutputhdfleswith. InthisworkMatlabisusedtoproducearedetectionproduct. Thecalibrateddatacanbeusedforfurtherlevel2processing.Oneex- Thelaststepisthecalibrationofthedatatheprogramcalibrateand

23 4FireDetection Intherstpartofthissectionthealgorithmsusedfordetectinganyres aredescribedtogetherwithassociatedprocedureslikedatainputandoutput whileinthenextone(fromsection4.4onwards)theimplementationofthese 4FIREDETECTION 23 integers.thetaskistoassignaphysicalmeaningtothesevalues. DataintheHDFlesproducedbyIMAPPisstoredin16bitunsigned algorithmsisexplained. areimportant.tousethefullpossiblerangeof16bit,ascaleandaosetis used.thethirdattributestoresthephysicalunit. example,alongnameofthedatasetcanbestored.herethreeattributes 4.1AlgorithmsforPreparingFireDetection EachscienticdatasetinaHDFlecanhaveassociatedattributes.For Thesameformulaappliesforreectancevalueswiththemodicationthat canbecomputedasb=(x?o)s scales,osetoandthedataxfromthehdfleareknown,theradianceb TheunitofradiancevaluesisusuallyinWm?2m?1steradian?1.If values,theyneeddatainmeansofbrightnesstemperature.brightnesstemperatureisthetemperaturewhereablackbodywouldemitthesameamount Butmostalgorithmsforredetectiondonotworkdirectlywithradiance m2msteradian: W reectancedoesnothaveaphysicalunitassociated. wherec1rstradiationconstantc1=2hc2=1: ?16wm?2 ofradiationforthegivenvalue. Planck'sformulaisusedasfoundin[12]or[2]. Toconvertradiancevaluestobrightnesstemperature,theinversionof C2secondradiationconstantC2=hck?1=1: ?2mK Tbrightnesstemperature(K) Bradiance( m2msteradian) T= Wln(C1=(5B)+1) C2= centerwavelengthofchannel(m) hplanckconstant cspeedoflight kboltzmannconstant

24 4FIREDETECTION ValueMeaning 0 ShallowOcean(Oceanlessthan5kmfromcoastORlessthan50m Table3:Possiblevaluesforland/seamask Land(notanythingelse). OceanCoastlinesandLakeShorelines. ShallowInlandWater(InlandWaterlessthan5kmfromshoreOR Ephemeral(intermittent)Water. DeepInlandWater(Inlandwatermorethan5kmfromshoreline lessthan50mdeep). likein[12]orappendixc.2. 76IfBisstillinW=m2msteradianafactorof106hastobeinserted ModerateorContinentalOcean(Oceanmorethan5kmfromcoast DeepOcean(Oceanmorethan500mdeep). ANDmorethan50mdeepANDlessthan500mdeep). ANDmorethan50mdeep) EvaluationoftheLand/SeaMask Theland/seamaskisprovidedbyIMAPPina1kmresolutionandcanhave Snowhasaveryhighreectanceinmanybandsandpossibilitiesareto thevalueslistedintable3asmentionedin[22].onlypixelswithaland/sea confuserepixelswithpixelscontainingsnow.evenifsnowandiceisnot maskvalueof1(land)or2(landwithcoastline/shoreline)shouldbetested forres,allotherpixelsareignored CreationandEvaluationoftheSnowMask commoninmostpartsofaustralia,thesnowymountainsarecoveredin index),butusesthebandscenteredat555nmand1640nm1insteadofred usedinthisworkarebasedon[13]. mostoftheyears. datafrommodisinstruments.itisthebasisof[17]and[13].thethresholds (NDSI)isutilized.ThisindexhasbeentestedwithdatafromLandsatand 1MODISchannels4and6 TheNDSIworkssimilartotheNDVI(normalizeddierentialvegetation Tondsnowpixelsintheimage,theNormalizedDierentialSnowIndex

25 andnearinfrared: 4FIREDETECTION Thesnowmappingconsistsofthreetests. NDSI=0:555?1:64 0:555+1:64 25 Thetwolattertestspreventverydarkpixelstobemappedassnow.Onlyif allthreetestsarepassedthealgorithmconsiderstheareaassnowcovered. NDSI>0:4 0:858>11%(band2) 0:555>10%(band4) taken,theywouldberegardedasres.in[1]alotofeortisundertaken anyre CloudDetection Similartosnow,cloudshavelargereectancevaluesandifnocarewouldbe Theuseofthecreatedsnowmaskisthatonsnowpixelstherecan'tbe approachwaschosen. highthickclouds.similarin[4]atleasteightdierenttestfordierentcloud typesaredone.herethisdierentiationisnotnecessaryandsoadierent testaredone: toclassifydierenttypeofcloudslikethincirrus,cloudsoversnoweldsor 0:488>95%(band10) 0:531>95%(band11) 0:551>95%(band12) Asmentioned,cloudsreectmanywavelengthsverygood.Soagainthree Ifandonlyifallthreetestsarepassed,apixelisdeclaredcloudyandno simplestformcomparesthebrightnesstemperaturearound4mwithagiven redetectionisdone.thetestwasvalidatedwithdayimagesfrommodis Thereareadierentapproachestodetectresinremotesensingdata.The only. atnight)butthisisnotthecasefortheareainsouthaustralia. threshold.thisworkswelliftheremainingimageisrelativelycool(especially 4.2FireDetectionAlgorithms

26 thebackground.kudohdescribesin[14]theuseofnormalizeddierence indicesbetweennoaaavhrrchannels3a,2and1. betweennoaachannels3and4. 4FIREDETECTION Otherworkslike[6]usearatiobetweentheAVHRR3:7mchanneland In[16]alsotwotraditionalthresholdtestareemployedplusadierence 26 timenotavailablefornearrealtimeapplicationslikethiswork. methodsliketheneuralnetworkapproachin[18]needmuchcomputation tests,comparisonswithbackgroundsandthresholdtests. MODIS.Additionallythesexedthresholdvaryseasonally.Someother problemswiththedierenttypesecosystemfoundinlargeimagesfrom Hereaalgorithmsimilartotheonein[15]isused.Itcontainsdierence Eachtechniquehasitsdisadvantages.Fixedthresholdmethodshave thenareisassumedregardlessoftheresultsofothertests.evenifthe reonthegroundismuchhotter,theeectivebrightnesstemperaturerarely channelsusedis1km,somostresaresmallerthanapixel.secondthe reachesthisvalue.thishasdierentreasons.thespatialresolutioninthe Thesimplesttestisaxedthresholdtest.If radianceisscatteredintheatmosphereandoftenobstructedbythesmoke T3:9>360K (1) plumefromthere. butthisisonlyanintermediateresult.thenthedierencebetweenchannel 21(3:9m)andchannel31(11m) Nextthebrightnesstemperatureiscomparedtoanothervalue iscomputedandcomparedtothexedthreshold T=T3:9?T11 T3:9>325K (2) T3:9bandTbtwofurthertestsaredone WiththebackgroundvaluesT3:9bandTbandtheirstandarddeviations T3:9>T3:9b+4T3:9b T>25K (3) Thevaluesarecomparedwiththeirbackground.Ifthedierenceislarger givenlocation. thanfourtimesthestandarddeviationtheteststatesapossiblereonthe T>Tb+4Tb (4) (5)

27 4FIREDETECTION istruethenthepixelisdeclaredtobearepixel.soithastopassoneof Ifthefollowinglogicalcombination Test1_((Test2_Test4)^(Test3_Test5)) 27 othertests. valuesaresuggested.thevaluesof360k,25kand325kshouldbereplaced, the3:9mtestsandoneofthettests.additionally,ifitpassesthebig thresholdtest(1)itisdeclaredarepixelregardlessoftheoutcomeofall 4.3AlgorithmsusingtheResultsofFireDetection accordingto[15]with330k,10kand315k.sinceuntilnownonightpassis available,thesevaluescouldn'tbeveriedinanyway. Fornightpasseswhichmayberecordedinthefuture,otherthreshold undistortedmapwithrepixelsmarked MapProjection Asexplainedinsection2.7,datafromMODISissubjecttotheBowTie Withtheresultsoftheredetectionathand,nextstepsarethevisualizationoftheseresults.Thegoalisrstalistofallresfoundandseconda dataintocommonformat,therectangularlatitude/longitudeprojectionwas eectandafewotherartifacts.toremovetheseeectsandtoputthe chosen.withthisprojectionalllatitudeandlongitudelinesareassumed introducesdistortions(aseveryprojectionfromaspheretoaplanedoes,see [27])intotheimage.Thefurtherapartfromtheequator,themoreexpanded parallelandofequaldistance.ofcoursethisisnottruesothisprojection thedataget.tobalanceoutthiseectalittlebitthefollowingassumptions aremade: Thedistancebetweenlongitudelinesis Thedistancebetweenlatitudelines(setofpointswiththesameinteger sucientforthisapplication. latitudevalue)is111.2km.thisnotexactforeverypointonearthbut waschosentobe34.86degreessouthasthisisthepositionofthe sucientaccuracyhere. receivingdishinmawsonlakes/adelaide. fortheentireimage.thisalsoassumesasphericalearthbuthasa 111:2kmcos

28 partsnorthofadelaideandexpandedforpartssouthofadelaide.another Theseassumptionshavetheconsequencethattheimagegetscompressedfor 4FIREDETECTION Thesizeofthesmallestpixelsinthenominal1kmresolutionisexactly 1kmx1km.Thisisusuallyonlytrueforpixelsalongthenadirline. 28 resultoftradingallimageswiththesameassumptionsisthattheseimages foundinthegeolocationdata.eachpixelintheconstructedgridhasa canbeconstructedbasedonthemaximumlatitudeandlongitudevalues canttogethertoformabiggermapifdesired. andawidthof heightof Withthenowknowndistancebetweenlatitudeandlongitudelinesagrid (111:2km deg)?11km (111:2km deg)?11km pixelcos(adelaide)=0:010959deg pixel=0: deg 4.3.2Smoothing Thesevaluesareconstantforallimages.Themaximumshiftbetweentwo thisonepixel(orless)shiftdierentimagesttogether. dierentimagesisonepixelbecauseofdierentimageorigins.despiteof pixel: viewableimage. remainblack.theseblackspotshavetobelledwithdatainordertogeta imagestayblackbecausetheimageelementsizeattheleftandrightedge recordedelementgetsprojectedinthenalgrid,manyspotsinthisgrid ismuchlargerthan1km.infactaimageelementcangrowbasedonthe 55degreemaximumviewangleofMODISupto5x2km.Sinceonlytheone Thedrawbackofthedescribedprojectionisthatmanypixelsintheresulting eachpixeltobelledthemedianlterconsidersallitsneighborsaccording approachwastheuseofmedianrankorderingltersforeachcolorband.for Forlargerltersizesthisapparentlyintroduceserrorsbecausepixelsfaraway totheltersize,sortstheminvalueorderandselectstheoneinthemiddle. thenumberofcandidatesoftengetseven(like2).sincethelternowcan't fromtheonetobelledcanlargelyinuencetheresult.forsmallltersizes Twodierentwaysofllingtheseblackpointshavebeentried.Therst value1)ortoobright(iftheltertriesvalue2inthiscase). takethemiddlevaluetheimagesgetseithertoodark(ifthelterchoose sumofallnonzeropixelsintheltersizeisdividedbytheircount.the Thesecondvariantofpixelllingwaswithaaveraginglter.Herethe

29 Thebestresultshavebeenfoundwithanaveraginglterofsize3x3applied darkpixelsremain. (likeclouds)spreadoveralargearea.ifasmallerltersizeischosen,many disadvantageoflargerltersizesislikeabovetheinuenceofextremepixels 4FIREDETECTION Theremainingblackpixelscanbeeliminatedbyreapplyingthelter. 29 twice.smallersquareshapedltersizesdonotmakesensesincea1x1lter Thenextlargeroddsizedlter(5x5)alreadyintroducesmanymoreerrors. doesn'tchangeanythinganda2x2ltershiftstheimageinonedirection. 4.4ImplementationofFireDetection FortherealizationofthedescribedalgorithmsMatlab6Release12wasthe chosenlanguage.thishasseveralreasons: AlgorithmsareeasytoformulateinMatlab.Thewrittenprogramsare Matlabhasagoodsupportforallneededleformats,particularlythe Theresultingprogramsarehighlyplatformindependentandcanrun easiertounderstandthansimilarprogramsinforexamplec. inputformathdfandtheoutputformattiff. ManycommonexceptionsarecaughtbyMatlab.ForexampleadivisionbyzeroisnotuncommonwhilecomputingtheNDSI.Thiscauses Theexecutionspeedisverygoodwhiledoingmatrixoperationsand oneveryplatformsupportedbymatlab. sucientinallothercases. in64bitdoublevalues.thisisespeciallymemoryinecientfor1bitlogical arrays. AdisadvantageofMatlabisitsdemandtostoreeverydataforcomputations andnotaruntimeerrorwithprogramabort. AllcomputationsfromtheendoftheIMAPPrunaremainlydoneinone onlyasuppressiblewarningmessageandtheresultnan(notanumber) Thefunctionconsistsofthefollowingparts: oneingure5.therethemainpartsoftheimplementationcanbeseen. Matlabfunction.Theaimofthissectionistodescribetheimplementation firedetectioninsomedetails. ofthealgorithmsmentionedaboveandtherunningofthematlabscript 1.Readinginallnecessarydata,i.e.importingpartsoftheHDFles Asamplerunofthescriptproducesoutputwhichlookssimilartothe

30 4FIREDETECTION 4.computationofthemap 3.Theactualredetection 2.Executingsanitycheckslikedetectionofcloudsandwater 30 itsexecution.theentiresourcecodeofthescriptisinappendixc.1. isstoredinhdf(hierarchicaldataformat)les.forabetterdescription oftheleformatseesection2.6.3and[25].sincematlabhasanexcellent 5.Andnallytheoutputoftheresults. supportforthehdfformat,thefunctionsforreadingdata(sdsread.min appendixc.4)andassociatedattributes(attributeread.minappendixc.3 Nowfollowsamoredetailedlookatthesourcecodeofthefunctionand canbekeptshort. Therstmaintaskistheimportofnecessarydata.DatafromIMAPP matetheconversionfromradiancetobrightnesstemperaturethefunction perature,butvaluesinthehdflesarestoredasradiances.toauto- land/seamaskareread.errorcheckingisdonewhereapplicable. 11,12,21and31.Additionallythemetainformationlatitude,longitudeand tionisnecessary.sothescriptreadsinthemodisbands1,2,3,4,6,10, Thealgorithmforredetectionneedsdataintermsofbrightnesstem- Forthedierentalgorithmsanumberofbandsandsomemetainforma- whereradianceisexpectedinw=m2msteradianasfoundinthehdf result=radiance2teff(radiance,lambda) radiance2teff.mwaswritten.callingsyntaxis viahelpradiance2teffissupported. canbeconvertedatonce. entlybaddata.thisincludesreectancevaluesgreaterthanone,brightness turnedinkelvin.theparameterradiancecanbeamatrix,somanyvalues lesandlambdashouldbeinm.aftersuccessfulcompletionresultisre- temperaturesgreaterthanthesaturationvaluesofthesensor,valueswhere radiance2teffcomputedinvalidresultsorwherethebaddatamarkerwas insertedbyimapp(-999forlatitude/longitudevalues). Thenextstepintheredetectionscriptistheeliminationofappar- ThesourcecodeisfoundinappendixC.2.TheusualMatlabhelpfacility ofimplementingsuchalgorithmsinmatlabcanbeseeninlineslike onlyvaluesof1or2areregardedaspossibleplacesforres.theelegance whichcreatesalogicarrayofthenecessarysizecontainingtrueforallland pixelsandfalseforallothers. land=(landseamask==1 landseamask==2); Theevaluationoftheprovidedland/seamaskisdoneasdescribedabove,

31 4FIREDETECTION 31 readingfinished readdatafromdisk >>firedetection(filename); metainformation 1kmemissive 1kmreflective 250mreflective 500mreflective Figure5:Sampleoutputfromthescriptfiredetection convertingradiancestotemperaturevalues lookingforbaddata evaluatingland/seamask detectingsnow detectingclouds detectingfires eliminatingbaddata 4um 11um creatingrgbimage producingrectangularmap smoothing(1/2) computingbackgrounds testing finding numberoffirepixelsdetected:78 calculatingvectorsandcopyingdata onedegreelongitudeis pixelswide onedegreelatitudeis111.2pixelshigh writingimage smoothing(2/2) findingblackspots countpixelswehavegot filtering writingasciioutput alldone >> paintingfires red green blue

32 thenthereisdenitelynosnowatthispoint.additionallyoneoftheother ariseswhenbothvaluesarezero.thenthedivisorbecomeszeroanda divisionbyzerowarningoccurs.thiswarningcanbeandissuppressed. Theinvalidresultisnoproblembecausewhenbothbandvaluesarezero 4FIREDETECTION Aproblemduringsnowdetection(infactduringcomputingtheNDSI) 32 twotestsfailstoo. Thisisachievedwiththelter lter.theeectisasmothercloudmaskincludingcloudedges. isaarrayoflogicvalues,theltercouldbedescribedasa3by3logicor cloudmaskisbroadenedbyonepixel.thisisdonebyconvolutingthecloud maskwitha3by3matrixcontainingonlyones.becausethecloudmask Fortheredetectionthecomputationofthepixelsbackgroundisneeded. Theclouddetectionworksstraightforwardexceptattheend,thefound B@ andthenbackgroundvaluesarecomparedtothepixels.allthresholdtests theedgesareexcludedfromfurtherprocessing. Theproblemwithconvolutionltersisalwaysattheedges.That'swhy Thestandarddeviationofeachbandiscomputedforthewholeimage CA aredonewithdaytimevalues.anighttimetestwasnotdoneatall. sitionsinthenalmap.thisisthetaskofthefunctioncstretch,which angularlatitude/longitudemap. valuesisdoneinthesubfunctionstretch. inordertodisplaythem.tofurtherimprovetheimage,aclippingofhigher ThelatitudeandlongitudevalueshavetobeconvertedtoXandYpo- Therestofthefunctiondealsmainlywiththecomputationoftherect- takesthegriddingfactorasaninput.thegriddingfactoris111.2kilometer forlatitudevaluesand111:2kmcos(adelaide)forlongitudevalues.when Thevaluesinthevisiblebandshavetobecompressedintothe83bit map,theimageiswrittentodiskandthereportingtextleisgenerated.at ingpixelsarelledwithaveragesoftheirneighbors. themap. theconversionisnished,thered,greenandbluepixelvaluesarecopiedinto Notallpixelsinthenalmapgetlledwiththisprocedure.Theremain- Afterthedetectedrepixelsaremarkedwithredspotsinthesmothed

33 thispointthefunctionreturns. 4FIREDETECTION 33

34 5CONCLUSIONS doesallworkfromtherecordedrawletotheredetectionproduct. thenecessarysequencebythesmallshellscriptinappendixe.3.thisscript 5Conclusions Allprogramsmentionedandallalgorithmsdescribedarecalledandrunin 34 sionwasmadetosimplifytheprocessandkeepallcomputationononeplace. inanautomaticsystem.becauseslicereliminatesmostofthebaddata Additionallyitproveddiculttocoordinatethemanydierentprogramsof already,thelackoferrorcheckinginframesyncisnotthatadverse. thestpsystem,sometimeskeystrokesarenecessary.thisisnotdesirable InthisscripttheprogramFrameSyncisusedinsteadofSTPS.Thisdecidatedinanumberofways: Atrstavisualinspectionoftheresultingmapwasdone.Allres Theresultsoftheredetectionalgorithmshavebeenempiricallyvali- foundareonlandandnotobstructedbyclouds.allreswerenear NextstepwasthecomparisontosimilarpicturespublishedbyNASAin eitheravisiblesmokeplumeoravisiblerescar.howeversomesmoke lutionofmodis.insomecasesthesmokeplumeitselfprohibitsthe plumesdon'thavearepixelnearthem.thishasseveralreasons. Someresarejusttoosmalltobepickedupbythe1kmx1kmreso- [24].Sincetheavailableimageswerefourdaysapart,onlyaveryrough detectionofthecausingrebyeitherbeingdetectedasacloudorjust obstructingtheviewtotheground. LastthedatawascomparedtothedatapublisheddailybyDOLA[9] comparisonwaspossible,butitshowedasimilarcongurationofre orignited. pixels.somereshaveadvancedadistance,otherswereextinguished Allvalidationsdoneshowtheresultsareveryreasonable.Thelackofmore datapreventedamorestatisticalevaluationofthecorrectnessofthealgorithm. anothersensor.astatisticalanalysiswasnotdonebecauseoflackof res.thecomparisonlookspromisingandshowsmanysimilarresults. ontheirwebsite.therethenoaa-avhrrdataisusedtodetect Asampleoutputoftheworkcanbeseeingure6.Thefoundresin enoughdata. Dierencesarecausedbythedierentrecordingtimeandtheuseof thisgurearelisted(amongothers)ingure7.theimagewasrecorded on10/08/2001andshowsanareaneargulfofcarpinteriainthenorthern Territory.Firescarsandsmokeplumescaneasilybespotted.

35 5CONCLUSIONS 35 Figure6:Pictureofdetectedres

36 5CONCLUSIONS #inputfiles: #ResultsoffiredetectionFigure7:Listofdetectedres 36 #startingtime:25-nov :31:18 #/dd1/imapp_run/ter10aug fires.txt(thisfile) #outputfiles: #/dd1/imapp_run/ter10aug fires.tiff #finishingtime:25-nov :33:46 #/dd1/imapp_run/ter10aug m.hdf #/dd1/imapp_run/ter10aug geo.hdf #thefollowingfireswerefound: #latitudelongitudeposit-xposit-y pixelhigh: degreeslatitude pixelwidht: degreeslongitude imagerange: / to /

37 6Recommendation Thereareanumberofeldswherefurtherworkcanbedone.Theseinclude 6RECOMMENDATION Attempttoforecastreintensitylikein[21] 37 Themappinghasdisadvantages.Especiallywhenatemporalanalysis Itispossibletodetecttheamountofsmoketodrawconclusionsabout Firescardetectionisusefulfortheestimationoftheamountofburnt theres.zhanqingliet.al.suggestedin[18]theuseofneuralnetworks todistinguishbetweensmokeandclouds. isdesired,amoreaccurategriddingofthedataisnecessary.works like[28]showhowthiscanbedone. Theprogramslicer.cshouldbemodiedtosliceevenwhennobad AVHRRdata. dataisfoundatcertainlesizeslikeevery400megabyte.thiswould areaandtheamountofproducedco2.in[5]thisisshownwith WhenAquawillbelaunchedafewadjustmentsmustbedonetothe Furthervalidationmustbedone.EspeciallyDOLA[9]shouldbea FrameSyncprogramtoadoptchangesin preventsomelackofmemoryproblemsandspeedupprocessing.additionallythestorageofimagesmaybeeasier. goodresourceforthiswork. Processingofnightimageswasnottriedatallandmaypromisegood resultsinredetection(butnotinimagery).forthistoworksomeminorchangestoframesyncarenecessary.thefunctionfiredetection {anddierentspacecraftid. {sequenceinpnandrsdecoding alsohastobechangedabitbecausesomebandsarenotavailableat night.

38 ADIMPLE DimpleisadigitalimageprocessingsystemfromProcessSoftwareSolutionWollongong,Australia.Itisdesignedasageneralimagemanipulation andprocessingsystembuthassomeinterestingfeaturesspeciallyforremote 38 ADimple togreyscaleimages,3dplotsordierentimageratios.sincenormalized ratiosaresupported,thelatteronecanbeusedforproducingindiceslike NDVIorNDSI. sensingandsatelliteimagery. lutionandrankorderlters.specialltersareforexamplethede-striping ltersforsatelliteimages. Theimageenhancementparthasthestandardltersincludingconvo- ThegeneralfeaturesincludesomeimagetransformationslikefromRGB IOLwhichisasimpleprogramminglanguageforimagemanipulation. allwithgroundcontrolpointswhichhavetobesuppliedbytheuser.this isamanualprocessandwastherefornotseenasanoptionforthisproject. manually. Thesameistruefortheclassicationalgorithms.Theyhavetobetrained Thebuilt-inlterscanbeenhancedwiththeImageOperationLanguage Therectication,registrationandresamplingalgorithmsprovidedwork IMAPP.ToimporttheselesinDimple,Dimplehastoknowhowtoread them,sotheuserhastosupplyforexampletheresolutionoftheimageand HDFlesandexporttheminaformatDimplecanread.AppendixD.2 containsthesourcecodeofthismatlabfunction.dataisexportedinbinary ofdimpletoreadhdfles.toovercomethismatlabwasusedtoimport lescontainingnootherinformationthantheactuallycalibrateddatafrom ThemaindisadvantageinconjunctionwithMODISdataistheinability thenumberofbandstoimport. hassuppliedthisinformation.forexampleitispossibletotransferlatitude/longitudedataasrealnumbersandintegerdataasuint16. Dimplecanreadsuchbinaryleswithdierentdatatypesaftertheuser

39 BGLOSSARY BGlossary ASMAttachedSyncMarker;beginningofeveryCADU ASTRAAutomaticSatelliteTrackingResearchAntenna;receivingfacility 39 AVHRRAdvancedVeryHighResolutionRadiometer;Instrumentonboard CADUChannelAccessDataUnit;consistsofASMandCVCDU ByteEightbits,usedsynonymouslywithoctet. initr CCSDSCenterforComputationalScienceandAdvancedDistributedSimulation NOAAsatellites CVCDUCodedVirtualChannelDataUnit;consistsofVCDUandReed HDFHierarchicalDataFormat;usedfordatastorageinlevel1aandlevel1b DEMDigitalElevationModel ITRInstituteforTelecommunicationsResearch MODISModerateResolutionImagingSpectroradiometer;Instrumenton SolomonCode NDVINormalizedDierenceVegetationIndex NDSINormalizedDierenceSnowIndex NCSANationalCenterforSupercomputingApplications MPDUMultiplexedProtocolDataUnit boardterraandaquasatellites NOAANationalOceanicandAtmosphericAdministration;partofU.S. OctetEightbits,usedsynonymouslywithbyte. STPSSatelliteTelemetryProcessingSystem;SoftwarefromGSFC VCDUVirtualChannelDataUnit;transportentityforCCSDSpackets wordsixteenbitsofdata DepartmentofCommerce

40 CMATLABSOURCEFILESFORFIREDETECTION 40 CMatlabSourceFilesforFireDetection InthissectionallMatlablesusedwhileredetectionarelisted.Some additionalfunctions,whicharenotnecessarybutsometimeshelpful,canbe seeninappendixd. C.1firedetection.m functionf=firedetection(filename); %FIREDETECTIONdetectfiresusingdatain`filename` %firedetectionisalongerscript %fordocumentation,pleasereferto %thesourcecodeand %KendyKutzner: %"ProcessingMODISDataforFireDetectioninSouthAustralia", %2001,ITR,Adelaide %changehistory %08/2001to10/2001writtenbykendykutzner %11/2001documentedbykendy %12/2001speedupbyvectorizingmappingbykendy %12/2001furtherdocumentationbykendy %let'sstart starttime=now; %nofailureuntilnow %mostcommonerrorsarecatched. %ofcoursenotall. %probablythethemostseriuosnotcatched %errorisoutofmemory.inthiscasematlab %abortsthefunction.thisscripttriestoclearmemory %asearlyaspossible. f=0; %readallthedata disp('readdatafromdisk'); %atfirst,weneedsomeinformationaboutthedatain %the.1000m.hdffile disp('metainformation'); filename1=[filename'.geo.hdf']; %forobviousreasonsweneedthese: [latitude,stat1]=sds_read(filename1,'latitude'); [longitude,stat2]=sds_read(filename1,'longitude'); %therearenofiresonwater,somakeuseof %theland/seamask [landseamask,stat3]=sds_read(filename1,'land/seamask'); %tohavebettervisualcontrolwhenneededall %dataistransposedsoitcanbeviewedwith %imagesc(bandx) %additionally,theimwrite()functionwantsthe %datainthisdirectiontoproduceimagesinthe %'right'direction latitude=double(latitude'); longitude=double(longitude'); landseamask=landseamask';

41 CMATLABSOURCEFILESFORFIREDETECTION 41 %catchsomeerrors if(stat1~=0 stat2~=0 stat3~=0) f=-1; disp('can''treadlatitude/longitudeorland/seamask'); return; end %nowwereadinallthedatabandsweneed. disp('1kmemissive'); filename1=[filename'.1000m.hdf']; bandname='ev_1km_emissive'; [emissives,stat1]=sds_read(filename1,bandname); [scales,stat2]=attribute_read(filename1,bandname,'radiance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'radiance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1; disp('can''treademissivebands'); return; end offsets=double(offsets); scales=double(scales); %datain.hdffilesisstoredwithscaleandoffset %tomakebetteruseof16bitdatafields. %Weneedtheactualvaluessoconvertthem. band21=((double(emissives(:,:,2))-offsets(2))*scales(2))'; %transposethearraysforthereasonabove band31=((double(emissives(:,:,11))-offsets(11))*scales(11))'; %thesearraystendtobequitebig,sofreememoryifwecan clearemissives; %iflackofmemoryisstillaproblem,a'pack'statementcan %beinsertedonthecriticalpoints. disp('1kmreflective'); filename1=[filename'.1000m.hdf']; bandname='ev_1km_refsb'; [reflectives,stat1]=sds_read(filename1,bandname); %forreflectivevalues,differentscalesandoffsetsapply [scales,stat2]=attribute_read(filename1,bandname,'reflectance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'reflectance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1; disp('can''treadreflectivebands(1km)'); return; end offsets=double(offsets); scales=double(scales); band10=((double(reflectives(:,:,3))-offsets(3))*scales(3))'; band11=((double(reflectives(:,:,4))-offsets(4))*scales(4))'; band12=((double(reflectives(:,:,5))-offsets(5))*scales(5))'; clearreflectives; disp('250mreflective'); filename1=[filename'.1000m.hdf']; bandname='ev_250_aggr1km_refsb'; [reflectives,stat1]=sds_read(filename1,bandname); [scales,stat2]=attribute_read(filename1,bandname,'reflectance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'reflectance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1;

42 CMATLABSOURCEFILESFORFIREDETECTION 42 disp('can''treadreflectivebands(250m)'); return; end offsets=double(offsets); scales=double(scales); band1=((double(reflectives(:,:,1))-offsets(1))*scales(1))'; band2=((double(reflectives(:,:,2))-offsets(2))*scales(2))'; clearreflectives; disp('500mreflective'); filename1=[filename'.1000m.hdf']; bandname='ev_500_aggr1km_refsb'; [reflectives,stat1]=sds_read(filename1,bandname); [scales,stat2]=attribute_read(filename1,bandname,'reflectance_scales'); [offsets,stat3]=attribute_read(filename1,bandname,'reflectance_offsets'); [names,stat4]=attribute_read(filename1,bandname,'band_names'); if(stat1~=0 stat2~=0 stat3~=0 stat4~=0) f=-1; disp('can''treadreflectivebands(500m)'); return; end offsets=double(offsets); scales=double(scales); band3=((double(reflectives(:,:,1))-offsets(1))*scales(1))'; band4=((double(reflectives(:,:,2))-offsets(2))*scales(2))'; band6=((double(reflectives(:,:,4))-offsets(4))*scales(4))'; clearreflectives; clearscalesoffsetsbandnamefilename1 %hopefullynowwegotallthedataweneed disp('readingfinished'); %correctnessofvaluesuntilnowverifiedwithhdflook %fromnowon,weareonourown disp('convertingradiancestotemperaturevalues'); %calculatethemiddleofthechannelsonthefly disp('4um'); band21=radiance2teff(band21,( )/2); disp('11um'); band31=radiance2teff(band31,( )/2); disp('lookingforbaddata'); %iftheconversionaboveproducedcomplexvalues, %theargumenttolog()wasnegative.something %isseriouslywrong,wewanttogetridofthis. imags=imag(band21) imag(band31); %the4umchannelsaturatesat500k,the11umchannelat400k %soallvaluesabovearebad %sametoreflectivebands:valuesabove100%arebad baddata=find(band1>1 band2>1 band3>1 band3>1 band4>1... band6>1 band21>500 band31>400 imags... latitude==-999 longitude==-999); clearimags; disp('eliminatingbaddata'); fori=[ ] istr=num2str(i); evalstring=['band'istr'(baddata)=0;']; eval(evalstring); end %specialcaseforlat/long,becauseweneedthe %min()andmax()functionlater.ifwewouldinsert %zeros,oneofthemwouldgetconfused. %min()andmax()takecareofnans,sowecanuseit

43 CMATLABSOURCEFILESFORFIREDETECTION 43 latitude(baddata)=nan; longitude(baddata)=nan; disp('evaluatingland/seamask'); %wewantonly"land"or"landwithcoastline" land=(landseamask==1 landseamask==2); clearlandseamask; disp('detectingsnow'); %nextlinemayproducedivisionbyzerowarning. %thisisok,soturnitoff warningoff; ndsi=(band4-band6)./(band4+band6); %wewanttohearwarningsagain warningbacktrace; %thisisthesnowdetectionalgorithm snow=((ndsi>.4&band2>.11)&band4>.1); snowpos=find(snow&land); %nexttotheclouds disp('detectingclouds'); cloud=((band11>0.95*max(band11(:)))&... (band10>0.95*max(band10(:)))&... (band12>0.95*max(band12(:)))); %theaboveclouddetectionalgorithmisnotthebest %Togetallthecloudedges,webroadenthecloudmask %withasmoothingfilter cloud=conv2(cloud,ones(5),'same'); %bringcloudmasktologicarrayagain cloud=cloud&1; cloudpos=find(cloud&~(snow&land)); %nowtothemainpart disp('detectingfires'); %atfirstweconstructabackgroundfilter backgroundfilter=ones(7); %weareonlyinterestedinthesurrounding backgroundfilter([345],[345])=0; %normalizethefiltersoitdoesn'tdisturbdata backgroundfilter=backgroundfilter/sum(backgroundfilter(:)); disp('computingbackgrounds'); %itprovedconvinienttomarkbaddatawithnan band21(band21==0)=nan; band31(band31==0)=nan; t41=band21-band31; t4b=conv2(band21,backgroundfilter,'same'); t41b=conv2(t41,backgroundfilter,'same'); %byfiltering,theedgeofeachmatrixbecomesinvalid %soweconstructainvalidmask invalid=zeros(size(t4b)); invalid([1:6,size(invalid,1)-5:size(invalid,1)],:)=1; invalid(:,[1:6,size(invalid,2)-5:size(invalid,2)])=1; %nowwecanseewhynanwasthebetterbaddatamarker invalid=invalid isnan(t4b) isnan(t41b); %wecomputethestandarddeviationonlyforvaliddata dt4b=std(t4b(~invalid)); dt41b=std(t41b(~invalid)); %nextistheperformingofthedifferentfiretests disp('testing'); day=1;

44 CMATLABSOURCEFILESFORFIREDETECTION 44 night=2; %thenextlinehastobereplacedbytheactual %day/nightdetection time=day; %thethresholdvaluesarebasedonatbd-mod-14 threshold=[ ; ]; %nowthefivetests firetest1=(band21>t4b+dt4b*4); firetest2=band21>threshold(time,1); firetest3=(t41>t41b+dt41b*4); firetest4=t41>threshold(time,2); firetest5=band21>threshold(time,3); disp('finding'); %combiningofthetests fire=(((firetest1 firetest2)&(firetest3 firetest4))... firetest5)&~invalid; %firescanonlybeonland,withnoclouds,andnosnow firepos=find(fire&land&~snow&~cloud); %theactualfiredetectionisnowdone. %restofthescriptdealswithmapconstruction disp(['numberoffirepixelsdetected:'num2str(size(firepos,1))]); %testingisdone,sowecancleanupalittlebit clearfiretestafiretestbfiretestafiretestbfiretestc; cleart41t41bt4bdt4bdt41b; clearband2band6band10band11band12band21band31; clearfirecloudsnowbaddatainvalidlandlandseamaskndsi; %nowproduceoutputimage disp('creatingrgbimage'); rgb=stretch(cat(3,band1,band4,band3)); %thefollowinglineswereusedfordebugging. %Iftheyareturnedonagain,theyproduceanice %imagetoo. %imagesc(rgb); %rgb2=rgb; %rgb2(firepos)=1; %rgb2(firepos+size(rgb2,1)*size(rgb2,2))=0; %rgb2(firepos+size(rgb2,1)*size(rgb2,2)*2)=0; %rgb2(snowpos)=0; %rgb2(snowpos+size(rgb2,1)*size(rgb2,2))=0; %rgb2(snowpos+size(rgb2,1)*size(rgb2,2)*2)=1; %rgb2(cloudpos)=0; %rgb2(cloudpos+size(rgb2,1)*size(rgb2,2))=0; %rgb2(cloudpos+size(rgb2,1)*size(rgb2,2)*2)=0; %figure %imagesc(rgb2) %continuewiththerealstuff disp('producingrectangularmap'); disp('calculatingvectorsandcopyingdata'); %wedon'twanttoloosedata,sointhemiddleofthe %mapapixelshouldbe1kmx1km. %onedegreelatitudeshouldbe111.2kmeverywhere %sizeofonedegreelongitudeisdependentonlatitude %weuseaustralia/adelaide/mawsonlakes ourlatitude=-34.86; %the'0-'inthenextlineistherebecauselatitudesare

45 CMATLABSOURCEFILESFORFIREDETECTION 45 %countedfromsouthtonorthwhilematlabimagesfromtop %tobottom [nlat,latfac]=cstretch(0-latitude,111.2); disp(['onedegreelatitudeis'num2str(latfac)'pixelshigh']); %cos(latitude)istheshrinkingfactorforlongitudevalues %thisassumesasphericalearthwhichisperfectlyrightfor %thispurpose [nlong,longfac]=cstretch(longitude,111.2*cos(ourlatitude/180*pi)); disp(['onedegreelongitudeis'num2str(longfac)'pixelswide']); %matlabindicesstartwith1 nlat=nlat+1; nlong=nlong+1; %createthetargetmatrixtospeedupcopying recmap=zeros(max(nlat(:)),max(nlong(:)),3); %thenextlinescopythedatafromrgb(:,:,:)to %recmap(:,:,:)accordingtotheinformation %infromlatitude/longitude vector=sub2ind([size(recmap,1)size(recmap,2)],nlat(:),nlong(:)); recmap(vector)=rgb(:,:,1); recmap(vector+size(recmap,1)*size(recmap,2))=rgb(:,:,2); recmap(vector+size(recmap,1)*size(recmap,2)*2)=rgb(:,:,3); %becausenotallpixelsareofthesamesize,inrecmap() %areblackspots.wewanttofillthem. %thisoperationwillneedsomememory,socleanupagain clearvector clearcloudposrgbsnowpos clearband1band3band4 pack; disp('smoothing(1/2)'); recmap=rgbfilter(recmap); disp('smoothing(2/2)'); recmap=rgbfilter(recmap); %thecodetopaintalinegridmaybeinsertedhere %nextthingistomarkthefireswithbigredpixels disp('paintingfires'); nfp=cat(2,nlat(firepos),nlong(firepos)); fori=1:size(nfp,1) recmap(nfp(i,1)+1,nfp(i,2)+1,:)=[100]; end %ifyouwanttoseetheresult... %figure; %imagesc(recmap); %disp('writingimage'); %imwritetendstousealotofmemory %soclearupandconvertthedatabeforehand clearband1band3band4rgbrgb2cloudpos; recmap=uint8(round(recmap*255)); pack; imwrite(recmap,[filename'.fires.tiff'],'tiff'); %hereweare!

46 CMATLABSOURCEFILESFORFIREDETECTION 46 finishtime=now; %lastthingsistowritethereport %ifotherinformationareneededinthereport %itiseasytomodifyit disp('writingasciiouput'); msg=['#resultsoffiredetection'10... '#inputfiles:'10... '#'filename'.geo.hdf'10... '#'filename'.1000m.hdf'10... '#outputfiles:'10... '#'filename'.fires.tiff'10... '# imagerange:'num2str(min(latitude(:)))'/'... num2str(min(longitude(:)))... 'to'num2str(max(latitude(:)))'/'... num2str(max(longitude(:)))10... '# pixelhigh:'num2str(1/latfac)... 'degreeslatitude'10... '# pixelwidht:'num2str(1/longfac)... 'degreeslongitude'10... '#'filename'.fires.txt(thisfile)'10... '#startingtime:'datestr(starttime)10... '#finishingtime:'datestr(finishtime)10... '#'10... '#thefollowingfireswerefound:'10... '#latitudelongitudeposit-xposit-y'10... ]; %fopen()shouldnotfail.ifitfails,wecan'tdoanythinganyway fid=fopen([filename'.fires.txt'],'w'); fwrite(fid,msg,'char'); fireinfo=cat(2,latitude(firepos),longitude(firepos),... nlat(firepos),nlong(firepos)); ifisempty(fireinfo) fwrite(fid,'#none','char'); elsefireinfo=sortrows(fireinfo,[3412]); fprintf(fid,'%10.5f%10.5f%10.0f%10.0f\n',fireinfo'); end fclose(fid); %ifwereachthispointwearehappy! disp('alldone'); return; %----- %somehelpfulinternalfunctions %stretchingcolorinthergbimage functionres=stretch(inp); mi=min(inp(inp~=0)); tmp=inp-mi; ma=.3*max(tmp(:)); res=tmp/ma; res(res>1)=1; res(res<0)=0; return; %stretchinglat/longvaluestomatrixcoordinates function[res,factor]=cstretch(inp,s);

47 CMATLABSOURCEFILESFORFIREDETECTION 47 secondfactor=1; tmp=inp-min(inp(:)); factor=s*secondfactor; tmp=round(tmp*factor); tmp(isnan(tmp))=1; res=tmp; return; %thefilteringoutoftheblackpixels functionrecmap=rgbfilter(recmap); fs=1;%sizeofthefilter xy=size(recmap); disp('findingblackspots'); %firstweneedtofindwhichpixelscanbefilled recmapcopy=sum(recmap,3); recmapcopy=conv2(recmapcopy,ones(2*fs+1),'same'); %excludetheedgesagain%%necessary?? recmapcopy([1:fs+1xy(1)-fs-1:xy(1)],:)=0; recmapcopy(:,[1:fs+1xy(2)-fs-1:xy(2)])=0; cxy=find(sum(recmap,3)==0&recmapcopy~=0); %allcoordinatesincxycanpotentiallyfilled %withdata disp('countpixelswehavegot'); %howmanynonzeroneighborshasapixel? %weneedthisasadivisorintheaveraging divi=zeros(size(recmapcopy)); clearrecmapcopy a1=(2*fs+1); %inthenextlinethe'&1'convertsthedoublearray %inalogicalarrayofonesandzeros. %wethensumovertheseones. divi=conv2((recmap(:,:,1)&1),ones(a1),'same'); disp('filtering'); disp('red'); tmp=conv2(recmap(:,:,1),ones(a1),'same'); recmap(cxy)=tmp(cxy)./divi(cxy); disp('green'); tmp=conv2(recmap(:,:,2),ones(a1),'same'); recmap(cxy+xy(1)*xy(2))=tmp(cxy)./divi(cxy); disp('blue'); tmp=conv2(recmap(:,:,3),ones(a1),'same'); recmap(cxy+xy(1)*xy(2)*2)=tmp(cxy)./divi(cxy); return; C.2radiance2teff.m functionteff=radiance2teff(radiance,lambda); %RADIANCE2TEFFconvertradiancevaluestobrightnesstemparatures %inputvalues: %radianceinw/m^2/um/sterradasfoundinimapphdffiles %lambdainum %radiancecanbeamatrix %ouputvalues: %teffeffectivebrightnesstemperatureinkelvin %changehistory

48 CMATLABSOURCEFILESFORFIREDETECTION %MODISAirborneSimulatorResearchandDocuments %??/11/2001addedcomputationofc1andc2 %citinghttp://ltpwww.gsfc.nasa.gov/mas/masdug.html %??/11/2001documented %08/11/2001writtenbykendykutznerfromscratchusingcitationbelow 48 %T(L,B)=C2/L*loge(C1/(L5*B(L,T)*106)+1) %where, %equation.the %inverseequationisoftheform %or'brightnesstemperature'maybedonebyinvertingtheplanck %ConversionfromIRradiancetoPlanckequivalenttemperature %DataUserGuide %byliamgumley,paulhubanks,andedmasuokaapril1994 %T(L,B)=brightnesstemperatureindegreesKelvin, %C2=(h.c)/k= mK %B(L,T)=PlanckradianceinWm-2sr-1um-1 %l=wavelengthinmeters %C1=2.h.c2= Wm-2 %speedoflightinvacuum %Planckconstant h= e-34; %secondradiationconstant c2=h*c/k; %firstradiationconstant c1=2*h*c^2; %c2= e-2; %c1= e-16; c= ; %Boltzmannconstant radiance=radiance.*1e6; k= e-23; %voila! C.3attributeread.m function[res,f]=attribute_read(filename,sdsname,attrname); %bringwavelengthtom teff=c2./lambda./log(c1./(lambda^5.*radiance)+1); %bringradiancetow/m^2/m/sterrad lambda=lambda./1e6; %filenameisthenameofthe.hdffiletoopen.itisnotparsedin %anyway,soitmaycontainpathinformationanditmustcontain %[res,status]=sds_read(filename,sdsname,attrname) %asinthe.hdffile. %theextension.hdf %sdsnameisthenameofthedatasettobeopened.itmustappearexactly %ATTRIBUTE_READreadsinScientificDataSetsfrom.HDFfiles

49 CMATLABSOURCEFILESFORFIREDETECTION 49 %attrnameisthenameoftheattributetoread.itmustappear %exactlyasinthe.hdffile. %resresult.itmayhavemorethanonedimensionandmaynot %beoftypedouble. %statuscontains0aftersuccessfulexecution,~0otherwise. %changehistory %22/11/2001writtenbykendykutzner f=0;%nofailuresuntilnow %openthefile sd_id=hdfsd('start',filename,'read'); ifsd_id==-1%failure disp(['error:couldnotopenfile"',filename,'"']); f=-1; return; end %searchfordatasetandopenit sds_idx=hdfsd('nametoindex',sd_id,sdsname); sds_id=hdfsd('select',sd_id,sds_idx); ifsds_id==-1%failure outstring=sprintf('error:sds"%s"notfound\n',sdsname); disp(outstring); f=-1; return; end %searchforattributeandreadit attr_id=hdfsd('findattr',sds_id,attrname); [res,status]=hdfsd('readattr',sds_id,attr_id); ifstatus~0 disp(['attribute"'attrname'"notfound']); f=-1; return; end %closedatasetandfile stat1=hdfsd('endaccess',sds_id); stat2=hdfsd('end',sd_id); if(stat1~=0) (stat2~=0) disp(['warning:problemsclosingfile'filename]); end return C.4sdsread.m function[mat,f]=sds_read(filename,sdsname); %SDS_READreadsinScientificDataSetsfrom.HDFfiles %[res,status]=sds_read(filename,sdsname) %filenameisthenameofthe.hdffiletoopen.itisnotparsedin %anyway,soitmaycontainpathinformationanditmustcontain %theextension.hdf %sdsnameisthenameofthedatasettoread.itmustappearexactly %asinthe.hdffile.

50 CMATLABSOURCEFILESFORFIREDETECTION %resallthedata.itmayhavemorethantwodimensionandmaynot %writtenbykendykutzner %changehistory %25/10/2001writtenfromscratch %statuscontains0aftersuccessfulexecution,~0otherwise. %beoftypedouble. 50 %08/11/2001addedfileclosure f=0; mat=[]; sd_id=hdfsd('start',filename,'read'); ifsd_id==-1%failure disp(['error:couldnotopenfile"',filename,'"']); f=-1; end sds_idx=hdfsd('nametoindex',sd_id,sdsname); ifsds_id==-1%failure sds_id=hdfsd('select',sd_id,sds_idx); disp(outstring); f=-1; return; outstring=sprintf('error:sds"%s"notfound\n',sdsname); return; [dsname,dsndims,dsdims,dstype,dsatts,stat]=hdfsd('getinfo',sds_id); if(stat~=0) ds_start=zeros(1,dsndims);%createsthevector[00]wherewewanttostart ds_stride=[];%wedon'tskipanything ds_edges=dsdims;%andreadtotheend [mat,status]=hdfsd('readdata',sds_id,ds_start,ds_stride,ds_edges); ifstatus~=0 disp(['warning:problemsgettinginfofor'sdsname]); return if(stat1~=0) (stat2~=0) stat2=hdfsd('end',sd_id); %closehdffile stat1=hdfsd('endaccess',sds_id); disp(['warning:problemsreadingsds'sdsname]);

51 DAdditionalMatlabsourceles DADDITIONALMATLABSOURCEFILES D.1all500m2tiff.m Withthegrowingnumberofavailabledatalestherewereincreasingdicultiestokeeptheoverviewwhichlecontainwhichoverpass.Tosolvethese 51 red,greenandbluein500mresolution,stretchingcontrastineachofthem individuallyandcombiningthemtoargbimage. vertsall500mhdflevel1blesindirectorytotifflesbyreadingbands problemsthesmallfunctionall500m2tiff.mwaswritten.thefunctioncon- needofmemory,abouttwicethesizeofthelargesthdfle. TIFFlesarewrittenishardwiredinthecodeofthefunction. functionall500m2tiff(directory); HDFles.Theparametermusthaveatrailing'/'.Thedirectorywherethe Thefunctionrunsautonomouslyifnoerrorsoccur.Adisadvantageisthe Theonlyinputparameteristhenameofthedirectorycontainingthe %ALL500M2TIFF(directory) %Convertsall500mHDFlevel1bfilesindirectory %ThedirectorywheretheTIFFfilesarewritten %HDFfiles.Theparametermusthavetrailing'/'. %directory:nameofthedirectorycontainingthe %in500mresolution,stretchingcontrastineachof %themindividuallyandcombiningthemtoargb %image. %Caution:needslotofmemory(roughlytwicethesizeof %totifffilesbyreadingbandsred,greenandblue %ishardwiredinthecodeofthefunction. %functionsneeded: %IMPORTHDF %STRETCHthelargestHDFfileindirectory) formatcompact; fori=1:s(1) s=size(f); targetdir='/home/ingest/kendy/tiffs/'; f=dir([directory'*500m.hdf']); %19/11/2001documentedbykendy %changehistory %25/10/2001writtenbykendykutznerfromscratch [nir,red,green,blue]=importhdf([directoryf(i).name]); disp('reading..'); clearnir; disp('stretching..'); disp(['workingon'f(i).name'(size:'num2str(round(f(i).bytes/1024/1024))'m)']); [a,b,c,d]=fileparts(f(i).name);

52 DADDITIONALMATLABSOURCEFILES rgb(:,:,1)=uint8(round(stretch(double(red))*255)); clearredgreenblue; hdfml('closeall') whosrgb; disp('packing..'); rgb(:,:,3)=uint8(round(stretch(double(blue))*255)); rgb(:,:,2)=uint8(round(stretch(double(green))*255)); 52 disp('writingtiff..'); imwrite(rgb,[targetdirb'.tiff'],'tiff'); disp('clearingup..'); return; res=tmp/ma; res(res>1)=1; ma=.4*max(tmp(:)); tmp=inp-mi; end;pack; functionres=stretch(inp); mi=min(inp(:)); clearrgb; toexportdatafromimapptodimplethematlabfunctionhdf2binwas written.itexportsalldatabandsin1kmresolutiontoabinaryleandsome metainformationfromthegeolocationprocesstoanother.theselescan D.2hdf2bin.m TheDIMPLEsoftware(seesectionA)can'treadHDFlesitself.Asaway pixelcompressionaredone. functionf=hdf2bin(filename,latmin,latmax,longmin,longmax) hdf2bincancuttheimagealonglatitudeand/orlongitudevalues. datatothebinaryle.especiallynocorrectionsforthebowtieeector bereadbydimpleviaitsrawlesimportfacility. %status=hdf2bin(filename,latmin,latmax,longmin,longmax) Thisscriptdoesnodataprocessinginanyway,itjustexportsthelevel1b Tolimittheresultinglesinsize,anareaofinterestcanbeprovidedand %tobereadbydimpleorsimilarprograms %filenameisthebasenameofbothhdffiles.theextensions %.1km.hdfand.geo.hdfareadded. %allremainingparametersgivetheboundarieswherethedata %shouldbecutted.ifnocuttingisdesired,give %ReadsinHDF1km&geofiles,storesoutputinbinaryfiles %-90,90,-180,180asvalues %statusequal0aftersuccessfulexecution,-1afteran %andheight. %erroroccured. %Thefilenameoftheoutputfilesarecreatedwiththe %basename,thentheresolutionoftheimageaftercutting %andlalohe.binforthemetainformationlatitude,longitude %inpixelsandtheextensiondata.binfortheactualdata

53 DADDITIONALMATLABSOURCEFILES 53 %created02/10/2001bykendykutzner %modified03/10/2001bykendy %-nowoutputaretwofiles,onewithsingleprecision(lat,long&height) %theotherwithuint16(databands) %-bandsarenowinnumericalorder %( lo13hi14lo14hi % ) %-filenamenowwithresolutioninpixels %modified19/11/2001bykendy %-addeddocumentation %someconstants nanreplace=0;%shouldinvaliddatabereplacedbynan? rigidcut=1;%leavedataoutsidespecifiedregionornot? f=0;%nofailureuntilnow %openthegeo-file actname=[filename,'.geo.hdf']; sd_id=hdfsd('start',actname,'read'); ifsd_id==-1%failure disp(['couldnotopenfile"',actname,'"']); f=-1; return; end %readinginlatitude ds_data=readdsbyname(sd_id,'latitude',2,nanreplace); lat=ds_data; %readinginlongitude ds_data=readdsbyname(sd_id,'longitude',2,nanreplace); long=ds_data; %readinginheighinformation ds_data=readdsbyname(sd_id,'height',2,nanreplace); height=ds_data; %finishedwiththe.geofile hdfml('closeall'); %openthe1000m-file actname=[filename,'.1000m.hdf']; sd_id=hdfsd('start',actname,'read'); ifsd_id==-1%failure disp(['couldnotopenfile"',actname,'"']); f=-1; return; end %lat,longandheighthavethesamedimension,so %cuttingthesethree(lat,long,height)together [area,xmin,xmax,ymin,ymax]=coordcut(lat,long,latmin,latmax,longmin,longmax); nlat=lat(ymin:ymax,xmin:xmax); nlong=long(ymin:ymax,xmin:xmax); height=height(ymin:ymax,xmin:xmax); bands=single([]); bands=cat(3,bands,nlat);

54 DADDITIONALMATLABSOURCEFILES 54 bands=cat(3,bands,nlong); bands=cat(3,bands,height); %readthedataandcutitasearlyaspossible ds_data=readdsbyname(sd_id,'ev_250_aggr1km_refsb',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); ds_data=readdsbyname(sd_id,'ev_500_aggr1km_refsb',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); ds_data=readdsbyname(sd_id,'ev_1km_refsb',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); ds_data=readdsbyname(sd_id,'ev_1km_emissive',3,nanreplace); tmp=ds_data(ymin:ymax,xmin:xmax,:); bands=cat(3,bands,tmp); hdfml('closeall'); %cleanupalittlebit,maybeweneedmem cleartmp; cleards_data; %cutrectangularoralonglat/long? if(rigidcut) areaf=area(ymin:ymax,xmin:xmax); j=size(bands); fori=1:j(3); bands(:,:,i)=single(double(bands(:,:,i)).*double(areaf)); end; end; %nowwearereadytowritethedata: bandssize=size(bands); count=length(bands(:)); if(bandssize(3)~=41)disp('error:notabletoread41bandsofdata');f=-1;return;end; actname=[filename,'.',num2str(bandssize(1)),'x',num2str(bandssize(2)),'.lalohe.bin']; fid=fopen(actname,'w'); iffid==-1 disp('warning:couldnotopenoutputfile"',actname,'"'); f=-1; return; end; %writelat,long,andheightin32bitfloatprecision count=count-fwrite(fid,bands(:,:,1:3),'single'); fclose(fid); actname=[filename,'.',num2str(bandssize(1)),'x',num2str(bandssize(2)),'.data.bin']; fid=fopen(actname,'w'); iffid==-1disp('warning:couldnotopenoutputfile"',actname,'"');f=-1;return;end; %allsuccesivewritesinuint16precision %250mbands1,2 count=count-fwrite(fid,bands(:,:,4:5),'uint16'); %500mbands34567 count=count-fwrite(fid,bands(:,:,6:10),'uint16'); %1kmreflbands8,9,10,11,12,13lo,13hi,14lo,14hi,15,16,17,18,19 count=count-fwrite(fid,bands(:,:,11:24),'uint16'); %1kmemissivebands20,21,22,23,24,25 count=count-fwrite(fid,bands(:,:,26:31),'uint16'); %1kmreflband26 count=count-fwrite(fid,bands(:,:,25),'uint16'); %1kmemissivebands27,28,29,30,31,32,33,34,35,36

55 DADDITIONALMATLABSOURCEFILES 55 count=count-fwrite(fid,bands(:,:,32:41),'uint16'); fclose(fid); ifcount~=0f=-1; disp(['warning:notalldatawrittencorrectly(',... end;num2str(count),'elementsleft)']); return; % %helpfulinternalfunctions % %gettingsds_idbyname functionsds_id=getsdsidbyname(sd_id,sdsname) sds_idx=hdfsd('nametoindex',sd_id,sdsname); sds_id=hdfsd('select',sd_id,sds_idx); ifsds_id==-1%failure outstring=sprintf('warning:sds"%s"notfound\n',sdsname); disp(outstring); f=-1; return; end return; % %readalldatafromagivensd-set functionds_data=readallsdsdata(sds_id,dsndims,dsdims) ds_start=zeros(1,dsndims);%createsthevector[00] ds_stride=[]; ds_edges=dsdims; [ds_data,status]=hdfsd('readdata',sds_id,ds_start,ds_stride,ds_edges); return; % %computethecuttingmatrix function[area,xmin,xmax,ymin,ymax]=... coordcut(lat,long,latmin,latmax,longmin,longmax) iflatmin>=latmaxdisp('warning:latmin>=latmax');end iflongmin>=longmaxdisp('warning:longmit>=longmax');end lat(lat<latmin lat>latmax)=0; lat(lat~=0)=1; long(long<longmin long>longmax)=0; long(long~=0)=1; %areacontainsonly0or1,so %convertareatouint8tosavememory area=uint8(double(lat).*double(long)); yx=size(area); iflength(yx)~=2disp('dimensionerror');return;end %nowlookforboundaries xmin=yx(2); ymin=yx(1); xmax=0; ymax=0; fory=1:yx(1) ifmax(area(y,:))>0 ymin=min(ymin,y); ymax=max(ymax,y);

56 forx=1:yx(2) DADDITIONALMATLABSOURCEFILES ifmax(area(:,x))>0 endxmin=min(xmin,x); xmax=max(xmax,x); 56 %readanentiredatasetbyname end % sds_id=getsdsidbyname(sd_id,dsname); [dsname,dsndims,dsdims,dstype,dsatts,stat]=hdfsd('getinfo',sds_id); functionds_data=readdsbyname(sd_id,dsname,dimension,nanreplace) ifsds_id==-1f=-1;return;end if(dsndims~=dimension)%safetycheck end ifstatus==-1 [attr,status]=hdfsd('readattr',sds_id,attr_idx); ds_data=readallsdsdata(sds_id,dsndims,dsdims); attr_idx=hdfsd('findattr',sds_id,'_fillvalue'); attr=-999;%attributenotfound,usingdefault disp('attribute"_fillvalue"notfoundin"latitude"'); f=-1; disp(['warning:unexpectedformatin"'dsname'"']); return; if(nanreplace)ds_data(ds_data==attr)=nan;end%replacingfillvalueswithnan

57 EOTHERSOURCECODES EOthersourcecodes andslicer.sectione.3describestheshellscriptwhichcallsallotherprograms. 57 Thissectionlistsotherusedsourcecodes.ThisincludestheprogramsFrameSync TheentireFrameSync.clewouldbefartoolongsoonlythedierences E.1FrameSync.c section d57 <intf_dumpframedecoded=false;/*dumpframesfoundafterremovingpn-randomization*/ totheoriginaloneareshownbelow.thedierencelewascreatedbythe standardgnudiprogram.forexplanationofthechangespleasereferto <intoffbyonebyte=0; <intf_flywheel=false; 60d58 <intn0bits=0; <126,130d120 </*bykendy*/ <intfframed=1; 85,87d82 <puts 160,161d149 <fs_shortstart_sync_match2[16]; 283,307d270 <case'4': <("-4[filename]=dumpframetotheoutputfileafterremovingPN-randomization"); <#else <if((fframed= <{ <++i; <#ifdef_win32 <_S_IREAD _S_IWRITE))<0) if(option_type[i+1]==opt_type_other) f_dumpframedecoded=true; <#endif <{ <0644))<0) sprintf(tbuf,"couldn'topenoutputfile%s", open(option[i],o_rdwr O_CREAT O_TRUNC, open(option[i],o_rdwr O_CREAT O_TRUNC O_BINARY, <} <(void)myerror(1,"-4optionrequiresafilename"); < (void)myerror(1,tbuf); else break; option[i]);

58 EOTHERSOURCECODES ,514c476 </*if(bitshift>7)*/ <if(bitshift>15) --- >if(bitshift>7) 549,550c511 </*for(i=0;i<8;++i)*/ <for(i=0;i<16;++i) --- >for(i=0;i<8;++i) 555,573d515 </****version2ofmakesyncsearchtable********/ </*bykendykutzner*/ </*I'mworkingonbits16-31ofsyncpattern <becausethesecontainvalidbitsforanypossiblebitshift <ThevaluesarestoredinStart_Sync_Match2*/ <void <Make_Sync_Search_Table2(longpattern) <{ <inti; <for(i=0;i<16;i++) <{ <Start_Sync_Match2[i]=pattern>>i; <} <} <606,607c548 </*for(i=0;i<8;++i)*/ <for(i=0;i<16;++i) --- >for(i=0;i<8;++i) 646,727d586 </*Find_Start_Sync2writtenbyKendyKutzner*/ </*muchfasteronnoisedata,butnotasaccurateastheorginal*/ </*thisdoesn'tmatterbecauseanothercheckisdoneafterwards*/ <int <Find_Start_Sync2(fs_longstartsrch) <{ <inti,j,r; <for(i=startsrch;i<framesize;i++) <{ <r=cvcdu_raw_frame[i+1]; <for(j=0;j<16;j++) <{ <if(r==start_sync_match2[j]) <{ < offset=i; < bitshift=j; < return(1); <} <} <} <return(0); <} <

59 EOTHERSOURCECODES 749a609,610 >staticintf_flywheel=false; >staticintn0bits=0; 833c694 <if(!find_start_sync2(startsync)) --- >if(!find_start_sync(startsync)) ,911c765 </*bykendy*/ <else 1362d >} 914d767 <}/*endfor*/ <weaccept*/ <theslightchancetofindabadsyncmarkerinthedate <{ <Make_Sync_Search_Table2(ShortSync[0]*0x10000+ShortSync[1]); <n0bits=0;/*syncfailed,soresetconsecutivegoodcounter. 1474c1326 <if(vcdu_primary_header.virtual_channel_id== </*****************************/ <{ <write(fframed,(char*)cvcdu_raw_frame,framesize); 1485,1502d1336 </*****IfDumpingdecodedFrames*****/ --- >if(vcdu_primary_header.virtual_channel_id==ch_id_modis) if(f_dumpframedecoded) Ch_ID_MODIS&&VCDU_Primary_Header.SpaceCraft_ID==0x2a) </*****setupthenextframetocheck******/ /*0x2a==TERRA*/ <continue; <} <#endif <break; <lastword=cvcdu_raw_frame[shortframesize]; <CVCDU_Raw_Frame[0]=lastword; 1561,1574d1394 <if(!readdata((char*)&cvcdu_raw_frame[1],framesize)) <{ <#ifdefdebug <fprintf(stderr, <"bogustelemetrypacketsize(nblks=%i,nframesread=%li\n", <Max_Packet_Size) <continue; <#endif <Have_Start_Of_Packet=FALSE; <nblks,nframesread); /*addedbykendykutzner*/ <} if(end_of_prev_packet+start_of_packet> <

60 EOTHERSOURCECODES c1436 < if(imageinfo.fout&&f_flywheel) --- > if(imageinfo.fout) 1728,1730d1547 </*insertedbykendy*/ <Start_Of_Packet+=(Packet_Size+7); </*endinsert*/ 1737c1554 < if(imageinfo.fout&&f_flywheel) --- > if(imageinfo.fout) E.2slicer.c Theprogramslicerisexplainedinsection3.1,basedonctest.cwrittenby RickyLuppinoandshownbelow. #include<errno.h> #include<string.h> #include<stdlib.h> #include<sys/types.h> #include<unistd.h> #include<stdio.h> #include<stdarg.h> #definechunk1024*1024 #defineblk1024 #definefns300 intmain(intargc,char*argv[]) {unsignedcharbuf[chunk]; //unsignedcharres[blk]; charfilename[fns+1]; chartargetfile[fns+1]; chartmpfile[fns+1]; FILE*fp; FILE*of; intfileopen; intstart_chunk; intchunk_count; intcorrel[blk]; intwinval[blk]; inti,j,offset; intk; intcount_array[256]; inthighest; inthighest_v; inthighcor; if(argc<2){ fprintf(stderr,"usage\nslicer<filename>[<targetfile>]\n"); exit(1); }strncpy(filename,argv[1],fns); if(argc>2){ strncpy(targetfile,argv[2],fns); }else strncpy(targetfile,filename,fns);

61 EOTHERSOURCECODES 61 strncat(filename,".raw",fns); fp=fopen(filename,"rb"); if(fp==null){ fprintf(stderr,"couldnotopen%s\n",filename); exit(1); }snprintf(tmpfile,fns,"%s.slicer%i.tmp",targetfile,getpid()); chunk_count=0; start_chunk=0; fileopen=0; //looparoundanddochunksuntileof while(fread(buf,1,chunk,fp)==chunk){ for(offset=0;offset<blk;offset++){ j=offset; for(k=0;k<256;k++) count_array[k]=0; while(j<chunk){ count_array[buf[j]]++; j+=blk; }//findhighest highest=0; highest_v=count_array[0]; for(k=1;k<256;k++) if(count_array[k]>highest_v){ highest_v=count_array[k]; highest=k; }correl[offset]=highest_v; winval[offset]=highest; }//findmaxcorrelvalueinthechunkandprint highcor=0; for(i=0;i<blk;i++){ if(correl[i]>highcor) highcor=correl[i]; }chunk_count++; //printf("chunk%d=%d\n",chunk_count,highcor); fprintf(stderr,"."); if(highcor==blk){ if(fileopen){ fwrite(buf,1,chunk,of); }else{ of=fopen(tmpfile,"w"); if(of==null){ fprintf(stderr,"couldnotopentempfile%s\n", tmpfile); exit(1); }fileopen=1; fwrite(buf,1,chunk,of); start_chunk=chunk_count; } }else{ if(fileopen){ //closefile fileopen=0;

62 EOTHERSOURCECODES //printf("attempttorename%sto%s\n",tmpfile,filename); fclose(of); snprintf(filename,fns,"%s-%i-%i.raw",targetfile, printf("%s-%i-%i\n",targetfile,start_chunk,chunk_count-1); if(0!=rename(tmpfile,filename)) start_chunk,chunk_count-1); fprintf(stderr,"error:%s\n",strerror(errno)); 62 }}if(fileopen){ fclose(of); snprintf(filename,fns,"%s-%i-%i.raw",targetfile, //printf("attempttorename%sto%s\n",tmpfile,filename); printf("%s-%i-%i\n",targetfile,start_chunk,chunk_count); if(0!=rename(tmpfile,filename)) start_chunk,chunk_count); } }E.3Theshellscript }fclose(fp); fprintf(stderr,"\n"); return0; fprintf(stderr,"error:%s\n",strerror(errno)); Thissmallshellscripttiesitalltogether.Itcallstheprograms Slicer reformat FrameSync IMAPPconsistingof withalltheirnecessaryarguments.italsocanupdatesomelesneededby Matlabwiththefunctionfiredetection {geolocate {calibrate {L0toL1a IMAPP=$IMAPPDIR/geoimapp.csh #IMAPPDIR=/dd1/IMAPP_RUN rawle. #!/bin/bash IMAPPDIR=/dd2/IMAPP_RUN2 SLICER=/home/ingest/bin/slicer FS2=/home/ingest/bin/fs2 IMAPP.Theonlyparameterittakesisthenameofthe(freshlyrecorded)

63 EOTHERSOURCECODES MATLAB=/app/matlab12/bin/matlab WGET=/usr/bin/wget REFORMAT=/home/ingest/bin/reformat CP=/bin/cp MV=/bin/mv #weshouldcheckwritepermissioninthecurrentdirectory 63 #checkforcommandlinearguments #maybethenextfourlinesshouldappearin #startoperation $CPutcpole.dat/data/IMAPP/level1a/static $CPleapsec.dat/data/IMAPP/level1a/static #$WGETftp://acdisx.gsfc.nasa.gov/pub/.dbs/ancillary/utcpole.dat #aweeklycronjob $SLICER$1>$1.slicer foractfilein`cat$1.slicer`;do #$WGETftp://acdisx.gsfc.nasa.gov/pub/.dbs/ancillary/leapsec.dat echo"workingonfile$actfile"; #turnthebytesarround #callingstps #thiswouldnotbeneccesarywithfs2butwithstps $FS2-i$actfile.raw-o$actfile.ccsds; #IMAPPcallsthelevel0files.PDS, #initrtheyarecalled.ccsds #insomefuturethefs2linemaybereplacedwithaline $REFORMAT<$actfile.raw>$actfile.tmp #sodoit $IMAPP$actfile.ccsds; #checktheresultsofimapp #echofiredetection\(\'$imappdir/$actfile\'\) $MATLAB-nojvm-nosplash; ls$actfile*-l $MV$actfile.tmp$actfile.raw done

64 FCongurationFiles FCONFIGURATIONFILES Thissectionlistnecessarycongurationles,mostofthemfortheSTPS suiteofprograms. 64 EDOS_F2Snugget F2Snugget F.1socapps.cfg S2Sstps edos CHANNEL0 *END* F.2stps.cfg VERSION3.41 S2Fstps STATUS_LOG_FILE_DIR./my ERROR_LOG_FILE_DIR./my CC_MODE1 CC_ENGR_DISPLAY1 DISPLAY_STRINGMY--TelemetryFromFileToFile DSP_PROCESS_UP0 INPUT_BUFFER_SIZE DSP_REC_DIR_NAME./my DSP_RECORD0 DSP_SESSION_TIME_OUT30 DSP_SOCKET_SEND_DATA_HEADER1 DSP_DEVICE0 INPUT_DATA_TYPE0 DSP_SOCKET_SERVER_PORT8002 DSP_SOCKET_OUTPUT_ON1 CHECK_INPUT_BLOCK_CRC0 REC_DATA_MAX_FILE_SZ-1 REC_DATA_TIME_LIMIT-1 DLTP_FRAME_DATA_TO_FILE0 DLTP_UP1 DLTP_PROCESSED_DATA_DIR/home/kendy/stps/my DLTP_SOCKET_HOST_MACHINEingest DLTP_SOCKET_CLIENT_PORT8003

65 FS_frameLength1024 FS_bitReverseFrames0 DLTP_SOCKET_RECV_DATA_HEADER0 FS_lossOfLockBits0 FCONFIGURATIONFILES 65 FS_autoPolarity0 FS_bitFlipsPerMode03 FS_framesPerMode31 FS_framesPerMode23 FS_framesPerMode19 FS_framesPerMode03 FS_pnDecodeCCSDS0 FS_pnDecodeBeforeReverse0 FS_bitFlipsPerMode13 FS_noFlywheelFramesFlag0 FS_outputFramesState2 FS_bitFlipsPerMode20 FS_bitFlipsPerMode33 FS_bitSlipsPerMode03 FS_bitSlipsPerMode13 FS_bitSlipsPerMode20 FS_bitSlipsPerMode33 FSP1CF FSP2FC FSP31D CHECK_FRAME_CRC0 RSD_ENABLED1 FSP01A FS_fsPatternLength4 RSD_BITS_PER_SYMBOL8 RSD_MAX_CORRECTABLE_ERRS16 RSD_Mo112 RSD_POA11 RSD_VIRTUAL_FILL0 EXTERNAL_MODULES_INPUT_PARAMS_BEGIN RSD_INTERLEAVE4 RSD_MODE1 EDOSdiscardBadFrames1 EDOSoutputServerHostnugget _EDOS_BEGIN_ ESH_VER1 TGT_PORTAA RSDheaderOnly0 EDOSSocOutPort2005

66 EDOCclcwPortNum0 EDOSinFreq EDOSoutputCLCW0 EDOSuseUNIXtime0 EDOSoutputSync1 FCONFIGURATIONFILES 66 EDOSclcwMulticastAddress TERRAspecificPNDECODE1 EXTERNAL_MODULES_INPUT_PARAMS_END END_PARAMS_ F.3pk.cfg RS_INTERLEAVE=4 #Totalframelengthincludingsync,andRSsymbolsifpresent TOTAL_FRAME_LENGTH=1024 #RSinterleave,setto0ifRSsymbolsarenotpresent #allparametersmusthaveaspaceinthefirstcolumn #allcommentsmusthavea"#"inthefirstcolumn #ConfigurationFileFormat PKT_DIRECTORY=. #ForeachVCtoprocess,enterparameters,nospacesor #commentsareallowedinbetweenparameters. #Basedirectorywherepacketfilesareplaced #1=yes,0=no INSERTZONE_SIZE=0 CLCW_PRESENT=0 (VC2-40havebeencutouttoshortenthisdocument.) VCID=1 CRC_PRESENT=0 VCID=42 INSERTZONE_SIZE=0 CLCW_PRESENT=0 VCID=41 CRC_PRESENT=0

67 CRC_PRESENT=0 INSERTZONE_SIZE=0 CLCW_PRESENT=0 VCID=43 FCONFIGURATIONFILES 67 (VC44{62havebeencutouttoshortenthisdocument.) CRC_PRESENT=0 INSERTZONE_SIZE=0 CLCW_PRESENT=0 VCID=63

68 REFERENCES References [1]SteveAckerman,KathleenStrabala,PaulMenzel,RichardFrey,Chris Moeller,LiamGumley,BryanBaum,CrystalSchaaf,andGeorgeRiggs. AlgorithmTheoreticalBasisDocumentDisriminatingClear-Skyfrom 68 [2]RonaldE.AlleyandMaritJentoft-Nilsen.AlgorithmTheoreticalBasis DocumentforBrightnessTemperature.JetPropulsionLaboratory,April CloudwithMODIS.MODISCloudMaskTeam,November1997. [4]M.D.AndrewsandP.E.Ardanuy.Ageneralizedscenarioforclouddetectionusingmodis-n.InGeoscienceandRemoteSensingSymposium, [3]StuartEllisAM AnnualReportoftheSouthAustralian CountryFireService.SouthAustralianCountryFireService,June [5]L.L.Bourgeau-Chavez,P.A.Harrell,E.S.Kasischke,andN.H.F.French. [6]S.A.Christopher,MinWang,K.Barbieri,R.M.Welch,andShi-Keng 1990.IGARSS'90,pages1487{1489,1990. ecosystemsusingers-1sarimagery.ingeoscienceandremotesensing Thedetectionandinterpretationofalaskanre-disturbedborealforest [7]SouthAustralianTourismCommission.AboutSouthAustralia,2001. Symposium,1995.IGARSS'95,volume2,pages1246{1248,1995. [8]ConsultativeCommitteeforSpaceDataSystems.SpacecraftID Yang.Satelliteremotesensingofres,smokeandregionalradiative energybudgets.ingeoscienceandremotesensing,1997.igarss'97, pages1923{1925vol.4,1997. [10]DundeeSatelliteRecievingStationHomePage.DundeeSatelliteReceivingStationMODISdownlinkpacketformat. [9]DepartmentofLandAdministration.SateliteRemoteSensingServices- Firewatch-HotspotDetection,December wa.gov.au/newsite/noaafd/noaafd.html. ccsds/ccsds.id. List, ac.uk/modisformat.html.

69 REFERENCES [11]CRCforSatelliteSystems.ASTRA.InstituteforTelecommunication [12]LiamGumley,PaulHubanks,andEdMasuoka.MODISAirborneSimulatorLevel-1BDataUserGuide,April Research, [13]DorothyK.Hall,AndrewB.Tait,GeorgeA.Riggs,andVincentV. [14]JunichiKudoh.Forestredetectioninfareastregionofrussiawith nasa.gov/mas/masdug.html. [15]YoramKaufmanandChrisJustice.AlgorithmTheoreticalBasisDocumentMODISFIREPRODUCT.MODISScienceTeam,1998. SensingSymposium,1999.IGARSS'99Proceedings,volume2,pages 858{860,1999. regionofrussiabyusingnoaaavhrrimages.ingeoscienceandremote snow-mappingalgorithm.in1997ieeeinternational,volume:2, pages619{621vol.2,1997.igarss'97.remotesensing-ascientic Salomonson.AlgorithmTheoreticalBasisDocumentfortheMODIS Snow-,LakeIce-andSeaIceMappingAlgorithms,October1998. [16]K.Kawano,J.Kudoh,andS.Makino.Forestredetectioninfareast noaa-15in1998.ingeoscienceandremotesensingsymposium,1999. IGARSS'99Proceedings,volume1,pages182{184,1999. [17]A.G.Klein,D.K.Hall,andG.A.Riggs.Improvingthemodisglobal [19]LockheedMartinMissiles&Space.InterfaceControlDocument [18]ZhanqingLi,A.Khananian,R.H.Fraser,andJ.Cihlar.Automatic IEEETransactionson,39(9):1859{1870,September2001. VisionforSustainableDevelopment. detectionofresmokeusingarticialneuralnetworksandthreshold [20]LockheedMartinMissiles&Space.DirectAccessSystemUser'sGuide (ICD)DataFormatControlBookforEOS-AMSpacecraft(ICD-106). IS C. approachesappliedtoavhrrimagery.geoscienceandremotesensing, [21]J.Minardi,G.B.Marchisio,andR.P.Treder.Spatiallinearmodeling RemoteSensingSymposium,1999.IGARSS'99,pages290{292vol.1, foreos-amspacecraft(icd-107),november1998.is andforecastingofforestresacrosstheunitedstates.ingeoscienceand

70 REFERENCES [22]MODISCharacterizationSupportTeam(MCST).ModisGeolocationVersion2ProductFormat,March2001. LatestFilespecs%/MOD03.geolocation.fs.txt. //modis-xl.nascom.nasa.gov/pub/stig_temp/mlinda/www/ ftp: 70 [23]NASA.AquaProjectScience, [24]NASA.NASA'sVisibleEarth, [25]NationalCenterforSupercomputerApplicationsattheUniversityof [26]MashNishihama,RobertWolfe,DavidSolomon,FrederickPatt,Jerey Sensors/Terra/MODIS.html. Illinois.HDFUser'sGuideVersion4.1r4,December2000.ftp://ftp. ncsa.uiuc.edu/hdf/hdf/documentation/hdf4.1r4/users_guide. [28]RobertE.Wolfe,DavidP.Roy,andEricVermote.Modislanddatastorage,gridding,andcompositingmethodology:Level2grid.Geoscience Location:AlgorithmTheoreticalBasisDocumentVersion3.0.MODIS [27]ArnoPeters.DieneueKartographie/Thenewcartography.Friendship Press,1983. ScienceDataSupportTeam,August1997. Blanchette,AlbertFleig,andEdwardMasuoka.MODISLevel1AEarth [29]YilinZhao.Vehiclelocationandnavigationsystems.ArtechHouse, andremotesensing,ieeetransactionson,volume:36issue:4, 36(4):1324{1339,July AppendixB.