SCHOOLOFCOMPUTERSTUDIES RESEARCHREPORTSERIES UniversityofLeeds Report AutomaticDetectionofKeySignature usingnotedistribution

Transcription

1 SCHOOLOFCOMPUTERSTUDIES RESEARCHREPORTSERIES UniversityofLeeds Report95.23 AutomaticDetectionofKeySignature usingnotedistribution DivisionofArticialIntelligence,SchoolofComputerStudies K.C.Ng,R.D.Boyle&D.Coopery ydepartmentofmusic by July1995

2 theirlocalharmonic`weight'.theauthorsreportarule-basedsystemfor tonalitytendtousekeysignatureswhichareeectivelyameasureof Westernmusicalcompositionswrittenintheperiodofcommon-practice Abstract calresultsaheuristicsearchismadewithrulesderivedfromhigh-level musicalknowledgeinordertoestimatethekeysignature. data,thedistributionofnoteclassesisobserved.usingtheseempiri- theautomationofkeysignaturedetectionwhichmakesuseofthetotal durationofindividualnoteclasses.fromacollectionofexperimental Introduction IndividualmovementsofWesternmusicalcompositionswrittenintheperiodof common-practicetonality,andmanycontemporarypopularsongsandinstrumentalpieces,tendtobegovernedbyasinglekeycentredenedasthetonic key.theidenticationofthiskeycentreinvolvesthedetectionofatonicnote, higher-levelknowledgeisrequiredtodiscoverwhetherthekeyismajororminor.generallyspeaking,themodalityisclariedbyreferencetothe`harmonic weighting'ofamovement,especiallyitsopeningandclosingsections,theinitial articulationofharmonicunits,andthenalcadenceoftenprovidingimportant clues.alistenerwithperfectorwell-developedrelativepitch,andsomesense ofthemusicalstyleofapiece,canusually,aposteriori,discriminateitskey. usuallysimpliedbythepresenceofakeysignatureinnotatedmusic,though andthedeterminationofthemodalityofitsassociatedtriad.thisprocessis 1

3 Theauthors'interestinthedevelopmentofanalgorithmfortheautomatic detectionofkeysignaturesarosefromtheirworkintheeldofopticalmusicrecognition(omr)(ng&boyle1992,hewlett&selfridge-field1993{4, thedevelopmentofsucharoutine,arule-basedprocedurefortheautomatic icationusingcontextualinformationwasrequired.asapreliminarystagein incompletekeysignatures,anditwasfeltthatamethodofcorrectionorclar- Ng&Boyle1994).InsomecasestheimplementationofOMRmayproduce detectionofakeysignaturefromtheperformancedatastoredinastandard MIDIFile(SMF)wasdeveloped,simulatingonacomputerthehumancognitive processesoutlinedabove.inthiscase,however,thework'scentraltonalityis deducedfromthedistributionofthedurationsofnoteclasses(pitcheswiththe juxtaposedpassageswithdierentkeysignaturesto: cidentals.anexampleofgrammaticalconventionsisseeninthetendencyof aknowledgebaseofstylisticconventions.forinstance,thetendencyofearly samenameirrespectiveoftheoctavetheyappearin),andnouseismadeof classical-periodcomposerstoavoidkeysignatureswithlargenumbersofac- movetoakeysignaturewiththeleastnumberofpossibleaccidentals(e.g. movetoakeysignaturewhichhasthesametypeofaccidental(e.g.b FmajortoD[major(5ats)ratherthanC]major(7sharps). majortof]majorratherthang[major);and 2

4 Approach algorithm:thedeltatime(dt)andthenote-oninstruction.deltatimes,which format(midi1988)providestwotypesofinformationwhichareusedbythe Therule-basedalgorithm(whichassumesatwelve-notechromaticsystem)functionsbytakingtheoveralldistributionofnoteclasses(theindividualnotenames data,arecodedinoneoftwoformats:assmptetimecode,witheachdtbeing indicatethetimedelaybetweendiscretemidieventsintheserialstreamof independentofmusicalregister)playedinamidile.thestandardmidile monlyused).thedtformatismadeexplicitinthele'sheader,asistheinitial tempochangesarecodedas`meta-events'eitherinthesingletrack(type0midi tempo,allowingthecalculationoftheabsoluteinter-eventdelay.subsequent amultipleoftheframeratetimesthesub-framerate;oraspulses-per-crotchet, les)orinaseparatetempo-changetrack(type1midiles). inwhichtheyrefertoadivisionofacrotchet(integermultiplesof96arecom- Thedurationofasinglenoteiscalculatedfromtheadditionofalldt'sfrom instruction,ormidinoteonwithzerovelocity).foreachnoteinthele, anote'sonset(midinote-oninstruction)toitstermination(midinote-o shownintable1representthetotaldelta-timesforeachnoteclassinthemidi durationdistributionofaperformanceofbach'spreludenumber1(frombook 1oftheWell-TemperedClavier(Bach1989)).Observethatthenotedurations ignoringitsoctave,thetotalduration(t)ofthenoteclass(n)beingplayed le.inthiscase,thepulses-per-crotchetformatisused,eachcrotchetbeing isconsideredastheweightofthenote.forexample,table1showsanote dividedinto240pulses. 3

5 Table1alsoshowsthenumberoftimeseachnoteclasswasplayed.Inthis comparetheincidenceofnotesa(50)andc(109)withtheirtotaltimesa asanoteoflongerduration(forexampleaminim).asanexampleofthis, quaver)needstobeplayedmanytimestoconstitutethesamelengthintime algorithm,wedonotconsiderthenumberoftimesanoteisplayedasitsweight (5128)andC(21096);hereadoublingofnoteincidenceinvolvesaquadrupling count.thisisbecauseanotewhichisshortinduration(forexampleasemi- oftotalduration.noteclassfrequencytotalduration C] D] EF(N) (T) F] G Table1:NotelengthdistributionofaperformanceofBach'sPreludeNo.1from Book1oftheWell-TemperedClavier. G] A] B Fromthetotalduration(T),thesortindexofeachnoteclassisassigned.The 6353 piece,thelowestsortindex(0)beingassignedtothenoteclasswiththelongest totalduration.usingdatafromtable1asanexample,thesortindexisas sortindexisarelativemeasureofthetotaldurationsofnoteclassesinthe illustratedintable2. 4

6 NoteclassTotalDurationSortindex C] D] EF(N) (T) (S) F] G] A] EmpiricalFindings BTable2:SortindexofTfromTable Twoofthemostsignicantnoteclassesofanykeyarethetonic(rstnoteofa Clavierhasatonicsortindexof3,alltheothershaveatonicsortindexof2or soundingnotes.infactonlyoneofthepiecesfrombach'swell-tempered valuesof3orless:inotherwordsthetonicisoneofthethreemostcommonly ExperimentallywefoundthatthesenoteclassestypicallyhaveSortindex(S) majororminorscale)andthedominant(fthnoteofamajororminorscale). less. Figure1showsahistogramofthesortindexof28MIDIles.Thesortindexof tonicsanddominantsofallthelesconsistentlyoccupiedthelowersortindex value. 5

7 illustration,fromthedataintable2,thefournotesclasses,asshownintable Bytakingthefournotesclasseswiththelowestsortindexvalue,wemayexpect tondthetonicandthedominantofthekeyamongstthesenoteclasses.asan Figure1:Sortindexdistribution (Largest Duration) Thesumsofthesortindicesofthetonicanddominantnoteclassesareused 3,are:C,G,DandE. 0(tonic) Semitone offset eachcandidateofthefourmostlikelynoteclassesastonic,thesumoftheir noteclassesaretypicallylowandnumericallyclosetoeachother.assuming tonicanddominantsortindicessum(st;sd)isfound.thenoteclassthathas inthenextstagetodeterminethekey.thesortindicesoftonicanddominant 6 Sort Index

8 NoteclassTotalDurationSortindex (N) GDCE (T) (S) 012 showsthevaluesofthesumsanddierencesoftonicanddominantnoteclasses thelowestsum(st;sd)istakenastherstestimationofthetonickey.table4 Table3:Thefourmostlikelycandidates. 3 ofthefourmostlikelycandidates. Note classtonicdominant (N)(St)(Sd)(Sum(St;Sd))(Diff(St;Sd))(Sum(St;Sd)+Diff(St;Sd)) G1 D2 C0 E3 12 Sum 13 SortIndices Di 1 Weight(W) Table4:Sumanddierencesoftonicanddominantsortindices Themajororminormodalitycanbedetectedsimplybycomparingtheindices dierencebetweenthesortindexoftonicanddominant(diff(st;sd)).inthis example,theinitialestimatedtoniciscwithaweightof2. Asmentionedabove,thetruekeyshouldhavealowSum(St;Sd)andalow beexpectedtohavealowersortindexfortheminorthirdaboveitstonicthan themajorthird,andamajorkeythecontrary.table5showsthesortindices ofthemajorandminorthirdsofeachofthefourmostlikelycandidatesand ofthemajorandminorthirdsabovetheestimatedtonic.aminorkeyshould 7

9 thesuggestedmodality Noteclass(N)Minor3rdMajor3rdModality GDCETable5:Determinationofmodality major minor RenedRules Insomecasesthetonicmaystillnotbeclear,fortherelativeminororthe followingrules: iscomparedwiththeotherthreelikelycandidates.tochallengetheinitial thedierenceintheirsum(st;sd)istypicallysmall.hencetheinitialtonic dominantmayberelativelystrongerinweight(lowernumericalvalue),though estimatedtonicandbecomethetonickey,acandidatetonicmustsatisfythe 1.Itsweight(Sum(St;Sd)+Diff(St;Sd))mustbenogreaterthentheestimatedtonic+2.Thisselectsnoteclasseswhoseweightingiscomparable tothatoftheestimatedtonic.musicallythisisbecauseweexpectthe tonicanddominanttobesignicantnoteclasses; thanthesortindexofitsminor7th+1; 2.Ifitisaminorkey,thesortindexofitsmajor7thshouldnotbegreater 8

10 3.LetUbethetotalduration(T)ofthetonicanddominantnoteclasses. IfUoftheestimatedtonic(UEstimated)isgreaterthanUofthecandidate 4.Itmustsatisfyatleastoneofthefollowingsub-rules: (UCandidate)by10%ofUCandidate,thecandidatecannotbetakenasthe newtonic. Thisisaheuristicruleadoptedafteranalysisoftheexperimentaldata; (a)thesum(st;sd)ofthecandidateandtheestimatedkeymustbe (b)thissub-rulehasfourparts:firstly,thecandidatetonic'sdiff(st;sd) similarweight. thatoftheestimatedkey. Musicallyweexpectthetonicanddominantnoteclassestohavea equal,andthediff(st;sd)ofthecandidateshouldbelessthan mustbelessthan4,tosatisfytheconditionthatthetonicanddominantshouldhavesimilartotaldurations; Secondly,thesortindexofthecandidatetonic'smediant(takenas anddominant. key;fromfigure1itcanbeseenthatthemediantmajorandminor (semitoneosets4and5)aresignicantnoteclassesafterthetonic theloweroftheindicesofthemajorandminorthirds)iscompared Thirdly,thecandidatetonic'sSum(St;Sd)mustnotbelargerthan totheestimatedtonic.itmustbelowerthanthatoftheestimated Lastly,thecandidatetonicmustbeoneofthetop3indices(bytonic noteclasslength){observedfromexperimentaldata. theestimatedkey+2(forthesamereasonasrule1); 9

11 (c)iftheestimatedtonicisminor,andthecandidatetonicismajor,and candidatetonicistakenasthenewestimatedkey. thecandidatetonichasalowerorequalsum(st;sd),theseventhof Aswithrule2,thejusticationforthisisthegeneraltendencyof theestimatedtonicistested.ifthesortindexoftheestimatedkey's major7thisgreaterthanthatofitsminor7th+1(thatis,theminor seventhofthescaleismoreprevalentthanthemajorseventh),the problemofthemis-detectionoftherelativeminorkey. minor7thnoteclass. Asortindexofminor7th+1waschosenasathresholdtoallowfor smalldierencesinthesortorder.thistestnormallycorrectsthe anon-modalminorkeytohaveamoreprominentmajor7ththan AprototypebasedonthealgorithmdescribedabovewaswrittenintheClanguageonaSiliconGraphicsIndyUnixworkstation.Itwastestedwithalarge particularcs.ruu.nl.forthe96individualpreludesandfuguesofbach'swell havebeenprocessedbythealgorithm,andextremelyhighsuccessrateshave TemperedClavierthesystemwas100%accurate.Manyothermusicalworks ExperimentalResults numberofmidilescollectedfromanonymousftpsitesontheinternet,in keyanditsrelativeminor.however,asboththesekeyssharethesamekey beenobserved. Characteristicallythealgorithmmayhavedicultydistinguishingbetweena 10

12 signaturethisisnotasignicantfailing.evenwithlarge-scaleformssuchas thosefoundinbeethoven'sfifthsymphony,thealgorithmcorrectlyidentied thekeysignaturedespitethepresenceofconsiderablemodulation. wereomittedfromtheperformance,theprogramincorrectlyresolvedthekey andtonicinbars19{21,andthesupertonicandmediantinbars29{34) signature,butwhenthetrillswereincluded,thecorrectresponsewasreturned. Inventions(inDminor).Whentwolongtrills(betweentheatleadingnote AninterestingndingwasassociatedwiththefourthofJ.S.Bach'sTwo-Part Thismaysuggestthattheseembellishments,farfrombeing`decorative',form ConclusionsandFuturedirections weightingfoundinthiscomposer'smusic. anintegralpartofthecomposition,andareessentialtothenormalnote-class asinglekeysignature,however,itshouldbepossibletoaccuratelyrecogniseit. tobesuccessfulwithworkswhichmodulatewidely,andinwhichnosinglekey Althoughthealgorithmhasproducedextremelypromisingresultsfromawide ThiswillbeparticularlythecaseforOMR,sincetheinputtothealgorithm selectionofpieceswrittenintheperiodfromaround1700{1850,itisunlikely mayberestrictedtoonepageofmusicalscoreatatime.itwouldbeinteresting toconsidersomecontemporarywesternartmusicworkswhichlackexplicitkey signaturedominatesthewrittenscore.withinindividualsectionsgovernedby signatures,yetwhichtheircomposersmaintaintobegovernedbyasinglekey centre,toseeiftheruleswehavedevelopedempiricallyalsosuggesttonal 11

13 ats,6sharps/6ats,and5sharps/7ats(inotherwords,keyswhich Thealgorithmdoesnotdistinguishbetweenkeysignatureswith7sharps/5 `centresofgravity'{onethinksespeciallyoftheworksofbelabartok(cooper 1996),forexampletheFourteenBagatellesforPianoOp.6(Bartok1908). MIDIspecicationwhichmerelygivesnotesanumber,butdoesnotdistinguishbetweensharporatforms.Thesemayberesolvedbyothercontextual areenharmonicallyidentical,andthussoundthesame).thisisduetothe information,andbyreferencetoconventionalpractice.allotherkeyscanbe detectedaccuratelyas`at'or`sharp'keys.theseambiguitieswillhavenoeffectontheapplicationtoomrsincetheoutputofsuchsystemswillexplicitly byomr.assuggestedintheintroduction,thesedatamaybeincomplete,and itisexpectedthataversionofthealgorithm,combinedwithaknowledgebase, containrecognisedaccidentalssymbols. shouldallowtheautomaticcorrectionofmistrackedorunresolvedkeysignatures.theknowledgebasewillmakeuseofsuchconventionsasthestandard Themostimportantextensionofthesystemwilldealwiththedataproduced layoutofkeysignature. Acknowledgements Theauthorswishestothankthecs.ruu.nlsiteforthemanyMIDIlesused forexperiment,anddr.yotomita(departmentofmusic,queen'suniversity, Belfast)forMIDIlesofBach'sWell-TemperedClavierBookTwo.Adistribu- 12

14 tionofthemidilesofbach'swell-temperedclavierbookoneandtwo,used forexperiments,canbefoundinftp.scs.leeds.ac.uk:/scs/doc/bachwtc[12].tar.gz, theanonymousftpserveroftheschoolofcomputerstudies,universityof Leeds. Bartok,B.(1908),FourteenBagatellesforPianoOp.6,Rozsnyai,Boosey& References Cooper,D.(1996),BelaBartok:ConcertoforOrchestra,Cambridge:CUP.To Bach,J.S.(1989),DasWohltemperierteKlavierI,HerausgegebenvonAlfred Durk.BWV846{869. Hewlett,W.B.&Selfridge-Field,E.,eds(1993{4),ComputinginMusicology, AnInternationalDirectoryofApplications,CenterforComputerAssisted Hawkes. ResearchintheHumanities,525MiddleeldRoad,Ste.120,MenloPark,CA appear. MIDI(1988),StandardMIDIFiles1.0,DistributedbyTheInternationalMIDI Ng,K.C.&Boyle,R.D.(1994),Reconstructionofmusicscoresfromprimitive Ng,K.C.&Boyle,R.D.(1992),Segmentationofmusicprimitives,in`ProceedingsoftheBritishMachineVisionConference,Leeds,UK.September1992', Springer-VerlagLondonLimited. Association.5316W.57thSt.,LosAngeles,CA subsegmentation,technicalreport94.12,schoolofcomputerstudies,the UniversityofLeeds.(ToappearinImageandVisionComputing). 13