Rameau: A System for Automatic Harmonic Analysis

Rameau: A System for Automatic Harmonic Analysis Genos Computer Music Research Group Federal University of Bahia, Brazil ICMC 2008

1 How the system works

The input: LilyPond format \score { << \new Staff { << \new Voice { \voiceone \relative c { c4 b c2 }} \new Voice { \voicetwo \relative c { e4 d e2 }} >> } \new Staff { << \clef "bass" \new Voice { \voiceone \relative c { c4 b c2 }} \new Voice { \voicetwo \relative c { c4 g c2 }} >> } >> }

The input: LilyPond format 3 4

The answer sheet format For the previous example: C G C For the 1st phrase of Bach Chorale #1 {g g c/e [b] d/f# g bm/d bm7/d em c [b d] [b] f# /a g d g d/f# em7/g f# /a g/b am7/c d d7 g}

The output as a table # answer ES-tree EC-Knn ES-Knn ES-PB S-PB ---------------------------------------------------- 1 C C C C C C 2 G G G G G G 3 C C C C C C ---------------------------------------------------- CORRECT(%) 100.00 100.00 100.00 100.00 100.00

The output as a score: Bach chorale #1 (Riemenschneider)

Comparison of results

Timing the program chorale #1 (80 segments) on my aging laptop: 14s for all 9 algorithms 0.2s for the fastest algorithm (neural network) 32s to analyze all 371 chorales using only the fastest algorithm

2 Problems

Pitch spelling equal not equal

Pitch class notation C=0, C =1, D=2, D =3,..., B=11 (or MIDI: 60, 62, 64... ) problem: C = D

A codification for tonal music (Dr. Jamary Oliveira) c d e f g a b 7 7 21 62 76 6 90 8 22 35 49 63 77 5 91 9 23 36 50 64 78 4 92 10 24 37 51 65 79 3 93 11 25 38 52 66 80 2 94 12 26 39 53 67 81 95 13 27 40 54 68 82 0 14 28 41 55 69 83 1 15 29 42 56 70 84 2 2 16 30 43 57 71 85 3 3 17 31 44 58 72 86 4 4 18 32 45 59 73 87 5 5 19 33 46 60 74 88 6 6 20 34 47 61 75 89 7 48 C=0, D=14, F =42, A =67

Segmentation (easy case) splitting the piece into chord spans

Segmentation (not easy) splitting the piece into chord spans

Chord labeling Labeling each chord with a name B 7 Eb D 7 Eb

Functional Analysis Assigning tonal functions to the chords in F major: F F C/E F/Eb Bb/D 6 4 6 I I V V /IV IV 2

3 The analysis per se

The process is: parse a score into a list of events and simplify it to a list of pitches

Each algorithm converts the list of pitches into chords

4 The algorithms

Pardo and Birmingham s algorithm straight template-matching 0 4 = Major 7 0 3 = Minor 7 And so on... (but we did extend it to have enough labels and be enharmonically savvy)

Temperley and Sleator s algorithm maximizing something over the piece Too slow. And too complex. And also too wrong, so we took it out.

Naïve Bayesian algorithm the chord that gives the highest probability to each sonority

Hidden Markov Model (Raphael) Bayesian, but also maximizes the probability of the entire piece in a Markov Model As you can see, it s the only one that can tell that a transition from Bm to D and then to Em is too unlikely.

K-Nearest Neighbors algorithm the most common chord label closest to the pitches being analyzed

Decision tree implicit rule-following If first pitch is 0 (a C) and second pitch is 28 (an E) and third pitch is 55 (a G), then the chord is a C major. If first pitch is 0 and second pitch is 27 (an E ) and third pitch is 55, then the chord is a C minor. And so on...

Neural networks (Tsui) pattern-matching over each sonority s pitches Currently the best algorithm is a neural network that only looks at one sonority at a time. Go figure.

5 Our Results

Average accuracies (10 chorales) Context-free neural net 95% ± 6 Context-free Knnn 94% ± 7 Context-considering knn 92% ± 8 Context-considering net 91% ± 10 Decision tree 89% ± 14 Naïve Bayesian 88% ± 19 Hidden Markov Model 83% ± 16 Pardo & Birmingham s 82% ± 21

Confusion matrix for the neural network M nc M7+ M7 m7 m ø7!!7 7 M x nc 2.8 x 1.8 1.8 M7+ 100 x x 4.2 M7 x m7 x m x ø7 7.1 x! x!7 50 50 x 7 x

Precision (%) Decision tree Contextual neural network M 93.7 94.2 nc 77.1 78.2 M7+ 50.0 100.0 95.7 100.0 M7 92.3 91.5 m7 72.2 93.9 m 97.2 95.5 ø7 91.7 76.9 7 100.0 66.7

Recall (%) Decision tree Contextual neural network M 96.1 97.0 nc 77.1 89.0 M7+ 33.3 16.7 91.7 83.3 M7 92.3 82.7 m7 78.8 93.9 m 96.3 98.2 ø7 78.6 71.4 7 100.0 66.7

F-measure (%) Decision tree Contextual neural network M 94.9 95.6 nc 77.1 83.4 M7+ 40.8 40.8 93.6 91.3 M7 92.3 87.0 m7 75.4 93.9 m 96.8 96.8 ø7 84.9 74.1 7 100.0 66.7

6 Computational musicology

Range Bach Kostk a Soprano Alto Tenor Bass

Range highest pitch lowest pitch soprano 116 254 282 298 331 334 186 034 050 068 075 223 049 070 100 165 185 197 214 325 348 158 175 205 239 352 087 110 244 alto 024 028 033 036 057 058 069 078 084 123 217 225 248 252 329 331 334 354 359 tenor 024 043 074 083 151 156 224 248 263 264 273 276 281 329 354 371 345 186 229 070 205 bass 285 312 331 034 051 214 223 340 131 328 143 155 197 205 219 241 187 235 319 337 240 070 075 165 175 239

Voice crossing (57% of Bach Chorales) 3 10 7

Consecutive octaves (4%) 5 7 10

Cadence detection: these are all the final cadences in the Bach chorales

Conclusion Pros free software: http://genos.mus.br/rameau algorithms from literature and new algorithms good accuracy easy performance evaluation quite fast :-) Cons no functional analysis no explicit segmentation no non-chord tone analysis only Bach Chorales natural ambiguity in harmonic analysis input data must be 100% correct