Can Statistical Language Models be used for the Analysis of Harmonic Progressions?

Transcription

1 Can Statistical Language Models be used for the Analysis of Haronic Progressions? Matthias Mauch 1, Daniel Müllensiefen #2, Sion Dixon, Geraint Wiggins # Centre for Digital Music, Queen Mary, Univ. of London, # Departent of Coputing, Goldsiths, Univ. of London 1 atthias.auch@elec.qul.ac.uk, 2 d.ullensiefen@gold.ac.uk ABSTRACT The availability of large, electronically encoded text corpora and the use of coputers in recent decades have ade Natural Language Processing (NLP) a flourishing research area. A wealth of standard techniques has been developed to serve use cases like docuent retrieval, identification of a finite vocabulary and synonys, and the collocation of ters. Siilarly, social networking aong usicians in internet forus and the advent of autoatic chord extraction have led to the establishent of chord databases, if on a saller scale. Coparatively little research has been carried out on these growing corpora of chords. We suspect that one reason for this lack of research lies in the difficulty to decide if chords or other haronic eleents can be treated like lexees in a text corpus. More siply, the question is: What is a word in ters of harony? In this paper we propose a botto-up approach. In order to find haronic units whose distributions reseble distributions of words we consider chord eleents differing in (a) length of chord sequence (counted in chord sybols), and (b) chord alphabet. Using lengths fro 1 to 4 and two different chord alphabets we obtain a paraeter space of size 8. For each of the paraeter settings we copute statistical suaries of the resulting frequency distribution of the haronic unit. As results, we report the paraeter settings for two different chord corpora (2500+ songs each) that generate a frequency odel corresponding ost closely to the Brown Corpus, a general text corpus of Aerican English. I INTRODUCTION Music and language ay be processed independently and in different parts of the brain, but soe obvious analogies relate the two doains, including evolution over tie, and the auditory syste as the priary gateway to perception. While there can be ultiple independent elodies at a tie in a piece of usic, chords have the particular property that positions the close to language: they are perceived sequentially. In fact, for ost tonal usic we can assue that at each tie point in a piece there is exactly one chord. Aptly, it is coon to refer to sequences of chords as chord progressions. The sequential nature of chord progressions has driven usic coputing researchers to use processing techniques known fro text and speech processing tasks. For exaple, hidden Markov odels siilar to those used in speech recognition have been successfully incorporated into autoatic audio chord labelling algoriths in order to achieve a sooth output (Bello and Pickens, 2005; Lee and Slaney, 2007). However, the odels are still uch sipler than their ancestors in speech recognition. Little attention has been paid to the proble of deciding what to odel: ost chord labelling algoriths assue that the nature of a chord is entirely deterined by its intervalic content, with no reference to its duration or its etrical position, i.e. ignoring haronic rhyth. In addition to that, it is (at least to us) very unclear whether just one chord is the haronic unit to odel and work with or whether progressions of two or even ore chords for a basic haronic unit which, for exaple, should be odelled as one state in a hidden Markov odel. Fro usic theory, it is known that one just a chord on its own (i.e. a root note and a chord type, e.g. d-inor) does not have a single function or eaning. Only in cobination with the intervals to surrounding chords, in relation to a current tonal centre, and in connection with the corresponding teporal data, distinctive haronic inforation eerges. In NLP any powerful techniques have been developed in recent years to solve tasks in text processing that have close analogues in usic processing. Aong these tasks are the clustering of docuents (or pieces in the case of usic) according to genre and style, the retrieval of ites fro large databases according to siilar content (query-by-exaple paradig), the retrieval of identical structures despite differing surfaces (e.g. for detection of plagiaris or cover songs), the induction of (latent) syntactic rules, or identification of word collocations or idios, i.e. lexees that are coonly associated. Many successful technical applications like internet search engines, speech recognition systes or electronic dictionaries incorporate solutions to these tasks. The techniques developed to tackle these probles include Latent Seantic Analysis, n-gra, hidden Markov and other probabilistic odels, statistics for word collocations, and probabilistic parsing and graar approaches. All these techniques operate on tokens or basic units into which a text or an utterance can be split. In any linguistic studies or applications, words are used as the basic token and the statis-

2 tical techniques are constructed or optiised to work on large corpora of texts that have words as their basic token (unit). Fro there follows the rationale of this study: If we want to eploy existing techniques fro NLP to tackle analogue probles in usic research using haronic data (e.g. clustering according to style or cover song identification) we would like to find a representation derived fro a large corpus of raw haronic data that has coparable distributional properties like those that are typically found in verbal corpora. In other words, out of the any possibilities of what could constitute a haronic word (i.e. the basic unit used for odelling) we ai at finding the one that generates a distribution that we can odel using standard ethodology fro coputational linguistics and that results in a odel siilar to those fro linguistic corpora. We have to stress that we consider the analogy to the concept lexee or word in language only in a distributional and aybe in a syntactical sense. We do, however, by no eans iply that a haronic word has any seantic quality coparable to the verbal unit. II TERMINOLOGY AND DATA Two of the ost iportant concepts of corpus linguistics are type and token. A type is a eber of the dictionary of a language, typically a word (or orphee), whereas a token is a eber of the saple, the instance of a type. For exaple, in an English language corpus the type that could describe tokens. As we deal with very different kinds of data, one has to bear in ind that a type in a language corpus will naturally be a word (or word for) whereas in a corpus of haronic data it will be soe unit of chord inforation, be it a chord or a chord sequence (see Section III for details). In fact, the ai of this study is actually to find out what a good type representation for haronic data ight be. A. Corpora We use two different popular usic corpora containing haronic (chord) inforation, which we call Counity Corpus and Autoatic Corpus. Each set features ore than 2, 500 songs. The Counity Corpus (2548 songs, chords) stes fro several sources and has been copiled by nuerous anonyous (aateur) usicians using the coercial software Band in a Box 1. The ain purpose of the progra is to generate a MIDI accopanient (in different styles) while taking as an input only a chord sequence (and optional elodies) provided by the user. Fro these Band in a Box files we extracted only the chord progressions, including etric duration. It is ipossible to test thousands of songs for transcription accuracy, but we assue that very bad quality files are rare in spite of the likely lack of professional usical skill because Band in a Box plays the generated transcriptions and users can check their files by ear. The Counity Corpus contains 1 a wide range of songs, including any Jazz standards, but also classic popular songs and folk songs. The chord data of the Autoatic Corpus (2592 songs, chords) set was autoatically extracted fro polyphonic MIDI files using the algorith proposed by Rhodes et al. (2007). The MIDI files used for this study was a saple fro a professionally assebled collection of 14,063 songs acquired fro the coercial MIDI distributor Geerdes MIDI Music 2 designed for professional use and karaoke playback. While the raw MIDI data is accurate, the autoatic extraction is likely to be noisy. This set also is very diverse but as the karaoke source suggests features ore conteporary and coercial pop usic. We copared these two usic corpora to the Brown Corpus, a standard and widely researched text corpus of written Aerican English published in 1967 which consist of 500 saples fro different conteporary text genres 3 (45,215 different words or types and 1,006,770 tokens or words in total). B. Data Forat, Chord Classes The Autoatic Corpus assigns to every beat (retrieved fro the MIDI representation) a chord label consisting of the chord root as well as the chord quality chosen fro a set of six labels aj, in, di, aug, sus9, sus4. We adopt this forat for the (originally richer) Counity Corpus and ap the chords appearing in it to these six classes 4. Chords that do not fit with any of the six classes are assigned to an auxiliary class called unknown. Both chord data sets are stored in RDF files using the Chord Ontology (Sutton et al., 2007), aking the the two largest chord corpora in open forat we are aware of. We plan to publish the Counity Corpus and the Autoatic Corpus later this year on the website III METHOD The analytical characterisation of the two pop corpora in coparison with the Brown Corpus akes extensive use of the zipfr package (Evert and Baroni, 2007) for lexical statistics within the R prograing language. We largely follow the analytical procedures proposed in the tutorial introduction of the package 5. While the text corpus data is already provided by the zipfr package, we have to copile statistics for the chord corpora fro the RDF files (see B.). In order to do so we load each chord corpus into our software and generate a suffix tree (for general inforation on suffix trees, see (Gusfield, 1997)). The suffix tree structure allows us to conveniently access uch of the inforation we need for further processing. We consider different alphabets of for a table see reduced.csv 5 severt/zipfr/aterials/zipfrtutorial.pdf

3 haronic eleents or haronic units which change with the paraeters described in this section. A. Paraeter Space While in texts words often have boundaries arked by space characters and the like, the chord sequence of a song provides us only with the inforation of chord changes. We look at four different kinds of haronic eleents, naely at single chords, and chord progressions of length 2, 3, and 4. Please note that only the first option is non-overlapping. We represent only the chord class of each chord (six different classes, see above) and the interval between successive chords. We assue that any song can be transposed into any key and still reains the sae. In uch the sae way as Mauch et al. (2007) we transpose any of the ites presented above so that the first chord has root C. Hence, two chord progressions such as C aj F in G aj and D aj G in A aj will be considered equivalent. Behind this procedure is the belief that the key of a piece is a concept that facilitates coposition and perforance but is less iportant for the listener (except for individuals with absolute pitch). Also, as reliable key inforation is not at hand, this is a convenient way to ipleent transposition invariance. As we want to investigate the influence of haronic rhyth, we included duration inforation, as both collections of songs contain inforation on the etric duration (quantised to beats) of the chords. In order to ake the inforation anageable, we apped the durations into three duration classes, naely 1beat, 23beats, and anybeats, where 1beat captures all chords that have a duration of one beat, 23beats those with duration of either 2 or 3 beats and anybeats those with 4 or ore beats. For exaple, if one distinguishes chords with different haronic duration, differs fro C aj - 1beat F in - anybeats C aj - anybeats F in - anybeats, while they obviously would not differ if one considered only the chord quality and root differences. The possible paraeter settings are hence to use etric durations or not to (and hence assue all chord durations are equal). Taken together, the 4 different length options for the chord progression cobined with the two types of alphabet generate 8 different paraeter settings that we explore in this study. Future investigations following this approach should also test the significance of etrical position of a chord in a bar, duration ratio between successive chords, the relation to the current key, and different chord class sets. This will, of course, increase the nuber of paraeter settings. B. Type Rankings The ost straightforward statistic of a text corpus is the ranking of types. The types we are considering are chord V Frequency Spectru Brown Corpus Figure 1. Word type frequency spectru of the Brown Corpus. is the frequency class, and V the corresponding nuber of word types. For exaple, the nuber of different words (word types) that occur = 2 ties in the corpus is about V 2 = n-gras on the one hand and words on the other. This representation has a value in its own right (Mauch et al., 2007) and can certainly serve to get an overview of the (trivial) characteristics of a corpus (Table 1). To illustrate how the haronic alphabet and the resulting distribution of relative frequencies differs between two paraeter settings, we picked two paraeter settings at rando, chord sequences of length 1 including etric duration and chord sequences of length 4 ignoring duration inforation, and list the along the relative frequencies of words in the Brown Corpus. C. Frequency Spectra The frequency spectra of linguistic corpora are characterised by the fact that any words occur only once or very rarely in the corpus while only a few words are used very frequently. The frequency spectru fro the Brown Corpus illustrates this nicely where we see the the frequency class = 1 being by far the class that includes the largest nuber of different words (V ), i.e. types. In Figure 2 one can see an epirical frequency spectru for chord progressions of length 3 in cobination with etric duration. The frequency spectru resebles the spectru obtained fro the Brown Corpus to a certain degree. We obtain a copletely different spectru fro the Autoatic Corpus corpus when coputed fro a representation of single chords (length=1) in cobination with the etrical duration inforation (see Figure 3): None of the haronic types appears only once and types occurring in the corpus have a uch higher frequency, i.e. are re-

4 Counity Corpus Brown Freq. rank etric durations, length 1 no durations, length 4 Corpus rel. freq type rel. freq type rel. freq type aj - anybeats aj 7 aj 5 aj the aj - 23beats aj 7 aj 7 aj of in - 23beats aj 7 aj and in - anybeats in 5 aj 5 aj to aj - 1beat in 5 aj 7 in 5 aj a di - 23beats aj 5 aj in in - 1beat in 5 aj 5 aj that 20 < sus9-23beats aj 2 aj at in 5 aj 9 aj if aj 11 aj way aj 2 in 8 aj 2 aj < hand 1000 < in 1 in 5 aj 6 in < charles < in 11 di 7 in 5 aj < registry Table 1. Type rankings and relative frequencies for two selected paraeter settings in the Counity Corpus as well as the Brown Corpus. For the chord sequences of length 4, the root difference between to consecutive chords is represented as an upwards interval easured in seitones, i.e. the chord change C-F would be an instance of aj. peated uch ore often. D. Productivity V Frequency Spectru: Autoatic Corpus, Reduced Chord Set, Metric Duration, Length Figure 2. Haronic type frequency spectru of Autoatic Corpus using chord sequences of length 3 and etrical duration inforation. Siilar in shape to Figure 1. A coon way to suarise these frequency spectra is to divide the nuber of words which only occur once (V 1, technical ter: hapax legoena) by the overall nuber of tokens in the corpus (N). The quotient is siply the proportion of types that have exactly one instance in the saple. It gives an indication of how the vocabulary is used and how productive the process is that generated the corpus, hence it is often called easure of productivity. Table 2 lists the productivity values for all paraeter settings in our study. Baayen (1994) explains in ore detail the use of the productivity easure in language. E. LNRE odelling The characteristic shape of the frequency spectra arising fro linguistic corpora can be odelled by so-called Large Nuber of Rare Events odels that allow us to suarise the frequency distribution fro a corpus by a few odel paraeters. Out of the several different odels applicable for this type of distribution we chose the finite Zipf- Mandelbrot odel as described by Evert (2004): { C π α 1 A π B g(π) = (1) 0 otherwise, where C = (1 α)/(b 1 α A 1 α ) is a noralising factor. What is odelled here is the density g(π) of types depending on their probability π. A quick intuitive explanation could go as follows: In a finite corpus the probability of a type (estiated by its relative frequency) never

5 Frequency Spectru: Autoatic Corpus, Reduced Chord Set, Metric Duration, Length 1 V V E[V ] Frequency Spectru observed fzm predicted Figure 3. Haronic type frequency spectru of Autoatic Corpus using chord sequences of length 1 and etrical duration inforation. One can see only a sall portion of the data, as ost types are concentrated in very high frequency classes, siilar to what can be observed in Table 1. Figure 4. Observed and predicted frequency spectru for the Brown Corpus fro finite Zipf-Mandelbrot odel falls below soe (sall) positive nuber. The paraeter A represents that nuber. Siilarly, the ost frequent type has a relative frequency soewhere between 0 and 1, and intuitively the density g(π) should be zero for values of π greater than that. Hence the use of B. The ost interesting paraeter however is α: the basic assuption of the odel is that there are any types which occur rarely, i.e. have a low probability. The density at values close to zero is therefore high, which is odelled by a power law, in which α characterises the slope of the type density curve. For a ore foral derivation, see (Evert, 2004). We fitted the finite Zipf-Mandelbrot odel to the Brown Corpus and to all 16 frequency spectra resulting fro paraeters the eight paraeter settings for each of the two corpora. We used the Siulated Annealing algorith for the odel fitting and paraeter optiisation. As visual exaples, we present graphs of frequency spectra as observed and as predicted by the finite Zipf- Mandelbrot odel for the Brown Corpus (α = 0.578; Figure 4) and two paraeter settings: Counity Corpus without duration inforation and length 4 (Figure 5), and Autoatic Corpus with etric duration and length 3 (Figure 6), as the latter two have α values (0.345 and 0.605) closest to that of the Brown Corpus. Looking at the observed frequency distributions and the corresponding odel predictions for these paraeter settings sees to suggest a reasonable good odel fit without any clear pattern of deviations between observed and predicted nubers of the first 15 frequency ranks. V E[V] Observed and Predicted Frequency Spectru Counity Corpus, No Duration, Length 4 observed fzm predicted Figure 5. Observed and predicted frequency spectru for the Counity Corpus fro finite Zipf-Mandelbrot odel. Paraeter setting: Length of chord progressions = 4, not using durational inforation.

6 Corpus Duration Length Exp. Voc. Size Max. Voc. Size Productivity alpha B at 100k tokens 1 No Duration 1 7 (7) No Duration (588) No Duration (49392) No Duration ( ) Counity 5 Metric Duration 1 21 (21) Metric Duration (5292) Metric Duration ( ) Metric Duration ( ) No Duration 1 5 (6) No Duration (432) No Duration (31104) No Duration ( ) Autoatic 13 Metric Duration 1 14 (18) Metric Duration (3888) Metric Duration (839808) Metric Duration ( ) Brown Table 2. Resulting bench ark values for all tested paraeter settings in coparison with the Brown Corpus. The Maxiu Vocabulary Size figures represent the theoretical vocabulary size possible by using the respective alphabet. IV RESULTS V E[V] Observed and Predicted Frequency Spectru Autoatic Corpus, Metric Duration, Length 3 observed fzm predicted Figure 6. Observed and predicted frequency spectru for the Autoatic Corpus fro finite Zipf-Mandelbrot odel. Paraeter setting: Length of chord progressions = 3, using etrical durations. Table 2 contains the results for all the different paraeter settings considered. We focus priarily on three bench ark values: The extrapolated expected vocabulary size at a corpus size of 100,000 tokens (i.e. for the Brown Corpus the vocabulary size after the first 100,000 tokens), the easure of productivity, and the α paraeter fro the LNRE- odel. For each of these bench ark values we look for input paraeter settings that generate values within a coparable range (i.e. roughly within the sae order of agnitude) to the Brown Corpus. Taking this rather qualitative look at the results table, we find the closest approxiation to the Brown Corpus for the etrical inforation setting of length 3 and the non-etrical setting of length 4. This is finding holds true for both the Counity Corpus and the Autoatic Corpus (see rows 2, 10 and 8, 16 of Table 2 respectively) V DISCUSSION We are aware that there are any ore paraeters that can be introduced into the current fraework. Also, we certainly know that there are any ore ways of looking at chords and haronic units that do not start fro a odel of (overlapping) n-gras. For exaple, one of the ost distinctive properties of usic, repetition (Huron, 2006), has been excluded in this paper. But we would like to include in a future extension of this study repetitionbased ethods for the segentation of a continuous sybol strea, like the algorith proposed by Cohen and Adas (2002). This would allow to cut the strea of

7 chord sybols into units of variable lengths. In addition and following the present approach, one could also consider reducing all chords to their roots (i.e. using only 1 instead of 6 chord classes) or on the contrary, extending the nuber of chord classes. Another option includes using a fixed tie window of, say, one or two bars, as a paraeter of the present fraework instead of a fixed length for the chord progressions. In addition to the present coparison with the word frequency odels distribution, looking instead at distributions of part-of-speech tags ight offer soe ore insight. VI CONCLUSIONS For this study we copiled and used the largest anually generated chord corpus we are aware of, as well as a chord corpus autoatically extracted fro MIDI data. We have identified two paraeter settings for treating chord inforation that result in a type frequency spectra that see to be coparable to a spectru fro a linguistic corpus and that can be odelled siilarly by a LNRE odel. One setting includes etrical duration and the other one akes no use of this durational inforation. We do not clai that this paraeter setting is the optial or ost adequate one for a haronic representation (this will be subject to further and ore rigourous testing and exploration of the paraeter space). But these settings allow us to look at the pop song corpora with a resolution coparable to the Brown Corpus. At this point it is not taken as granted that this resolution is adequate for investigation of the haronic content of pop usic corpora. Only when we use the resulting haronic units in an application (e.g. LSA for usical style clustering or cover song detection) will we be able to test whether the generation of haronic units actually has given useful and in that sense eaningful results. Nonetheless, the exploratory investigations presented in this paper are a necessary first step for pruning the enorous paraeter space that is potentially relevant when we ai at finding a reliable unit for haronic odelling. sequences. In Proceedings of JADT 2004, pages , Stefan Evert and Marco Baroni. zipfr: Word frequency distributions in R. In In Proceedings of the 45th Annual Meeting of the Association for Coputational Linguistics, Prague, Czech Republic, Dan Gusfield. Algoriths on Strings, Trees, and Sequences: Coputer Science and Coputational Biology. Cabridge University Press, David Huron. Sweet Anticipation: Music and the Psychology of Expectation. MIT Press, Kyogu Lee and Malcol Slaney. A Unified Syste for Chord Transcription and Key Extraction Using Hidden Markov Models. In Proceedings of the 2007 ISMIR Conference, Vienna, Austria, Matthias Mauch, Sion Dixon, Christopher Harte, Michael Casey, and Benjain Fields. Discovering Chord Idios through Beatles and Real Book Songs. In ISMIR 2007 Conference Proceedings, Vienna, Austria, Christophe Rhodes, David Lewis, and Daniel Müllensiefen. Bayesian Model Selection for Haronic Labelling. May MCM Berlin. Christopher Sutton, Yves Raiond, Matthias Mauch, and Christopher Harte. The Chord Ontology, URL References R. Harald Baayen. Productivity in language production. Language and Cognitive Processes, 9(3): , August Juan P. Bello and Jerey Pickens. A Robust Mid-level Representation for Haronic Content in Music Signals. In Proc. ISMIR 2005, London, UK, Paul Cohen and Niall Adas. An unsupervised algorith for segenting categorical tieseries into episodes. In Pattern Detection and Discovery, ESF Exploratory Workshop, London, UK, Septeber 16-19, 2002, Proceedings, Stefan Evert. A siple LNRE odel for rando character