Harmonic random walk in major

This is going to be a rather long post and may be interesting for music theory enthusiasts.

The program implements a random walk through all diatonic chords in all major keys, i.e, 12 x 7 = 84 chords. This is done with the help of a markovian matrix that defines the transition probabilities. However, the transitions are guided by rules in order to achieve a ‘good sounding’ result.

First, the transitions from key to key prefer neighboring keys where the major scales differ by only a few notes. The keys are organized in a ring containing the differences to a given midi base note:

keys = ring(0, 7, 2, 9, 4, 11, 6, 1, 8, 3, 10, 5)

If we assume 0 = C, then this sequence would be C-G-D-A-E-H-Gb-Db-Ab-Eb-B-F, which is a sequence of increasing fifth, or, falling fourth in the reverse direction.

Second, the transitions from chord degree to chord degree are organized in this ring:

ring(:i, :iv, :vii, :iii, :vi, :ii, :v)

Again, this sequence provides for falling fourth when ticking along. Neighboring chords share as many notes as possible.

Third, the probabilities in the markovian matrix restrict transitions from one key to another in such a way, that only specific chord transitions are used to change the key. For example, the line

markov[:i][[-2, :ii]] = prop_key_change[-2]

allows for changing the key by a -2 step in ring keys only if a :i chord is played in the previous key, and the transition results in a :ii chord in the new key. Example transition: Cmaj7 in key C → Cminor7 in key Bb. This mechanisms establishes ‘doors’ that the random walk will use when changing the key. And the used chords will be related to each other because they share some notes. Therefore, the key transition will ‘sound good’.

All probabilities can be set with parameters. The probabilities for stepping forward in ring degrees are set by map prop_deg_change, and the propabilities to change keys by stepping back and forth within ring keys are set by the map prop_key_change.

Example for a setting that realises a pretty standard ‘functional’ chord progression without key change. The chords just tick along by falling fourth. Such very smooth progressions have been used during the baroque period a lot:

prop_deg_change = {
  1 => 1.0,
  2 => 0.0,
  3 => 0.0,
  4 => 0.0,
  5 => 0.0,
  6 => 0.0
}
prop_key_change = {
  -3 => 0.0,
  -2 => 0.0,
  -1 => 0.0,
  1 => 0.0,
  2 => 0.0,
  3 => 0.0,
}

In contrast, the following setting implements a lot of key changes and also wilder jumps within the ring of degrees:

prop_deg_change = {
  1 => 0.3,
  2 => 0.2,
  3 => 0.1,
  4 => 0.0,
  5 => 0.0,
  6 => 0.1
}
prop_key_change = {
  -3 => 0.05,
  -2 => 0.05,
  -1 => 0.15,
  1 => 0.01,
  2 => 0.05,
  3 => 0.05,
}

When experimenting with these settings, a rule of thumb is that those propabilities roughly sum up to 1 or a bit less than 1. If the sum is less than 1, the remaining probability is used to simply repeat the same chord.

Here is the complete program:

# harmonic random walk in major
# written by Nechoj

use_debug false
use_random_seed 42

# chord degree sequence
degrees = ring(:i, :iv, :vii, :iii, :vi, :ii, :v)
# prepare hash map to get index in degrees if value is given
degrees_idx = Hash.new
degrees.each_with_index do |k, i|
  degrees_idx[k] = i
end

# key sequence: delta midi notes relative to a base key like :C3
# example when 0=C: C-G-D-A-E-H-Gb-Db-Ab-Eb-B-F
keys = ring(0, 7, 2, 9, 4, 11, 6, 1, 8, 3, 10, 5)
# prepare hash map to get index in keys if value is given
keys_idx = Hash.new
keys.each_with_index do |k, i|
  keys_idx[k] = i
end

# probabilities for changing the index within ring degrees
# prop_deg_change[n]: probability for transition degrees[i] -> degrees[i+n]
prop_deg_change = {
  1 => 0.4,
  2 => 0.2,
  3 => 0.1,
  4 => 0.0,
  5 => 0.0,
  6 => 0.1
}

# probabilities for changing the index within ring keys
# prop_key_change[n]: probability for transition keys[i] -> keys[i+n]
prop_key_change = {
  -3 => 0.0,
  -2 => 0.05,
  -1 => 0.15,
  1 => 0.03,
  2 => 0.0,
  3 => 0.0,
}

# markovian matrix = map of maps for transition probabilities
markov = Hash.new
degrees.each do |deg|
  markov[deg] = Hash.new
end

# set transition probabilities for key changes
# Interpretation of markov[:i][[-2, :ii]]:
# if last played chord was degree :i and in key keys[n], then
# with probability given by markov[:i][[-2, :ii]]
# change to chord with degree :ii played with key keys[n-2]
markov[:i][[-2, :ii]] = prop_key_change[-2]
markov[:i][[-1, :v]] = prop_key_change[-1]
markov[:i][[1, :iv]] = prop_key_change[1]

markov[:ii][[-3, :vii]] = prop_key_change[-3]
markov[:ii][[-2, :iii]] = prop_key_change[-2]
markov[:ii][[-1, :vi]] = prop_key_change[-1]
markov[:ii][[1, :v]] = prop_key_change[1]

markov[:iii][[-1, :vii]] = prop_key_change[-1]
markov[:iii][[1, :vi]] = prop_key_change[1]
markov[:iii][[2, :ii]] = prop_key_change[2]
markov[:iii][[3, :v]] = prop_key_change[3]

markov[:iv][[-2, :v]] = prop_key_change[-2]
markov[:iv][[-1, :i]] = prop_key_change[-1]
markov[:iv][[1, :vii]] = prop_key_change[1]

markov[:v][[-2, :ii]] = prop_key_change[-2]
markov[:v][[-1, :ii]] = prop_key_change[-1]
markov[:v][[-1, :v]] = prop_key_change[-1]

markov[:vi][[-2, :vii]] = prop_key_change[-2]
markov[:vi][[-1, :iii]] = prop_key_change[-1]
markov[:vi][[1, :ii]] = prop_key_change[1]
markov[:vi][[2, :v]] = prop_key_change[2]

markov[:vii][[-1, :iv]] = prop_key_change[-1]
markov[:vii][[-1, :vii]] = prop_key_change[-1]
markov[:vii][[3, :v]] = prop_key_change[3]

# set transition probabilities for chord changes without key change
degrees.each do |deg|
  this_idx = degrees_idx[deg]
  range(1, 7, 1).each do |i|
    markov[deg][[0, degrees[this_idx + i]]] = prop_deg_change[i.to_i]
  end
end

# finally set probabiltities for no change (play same chord again)
# adjust row sum of markov map equal to 1.0
degrees.each do |deg|
  cum_probs = 0.0
  markov[deg].each_value do |v|
    cum_probs += v
  end
  if cum_probs > 1
    puts "warning: probabilties greater than 1, normalizing", deg
    markov[deg][[0, deg]] = 0.0
    markov[deg].each_key do |k|
      markov[deg][k] /= cum_probs
    end
  else
    markov[deg][[0, deg]] = 1 - cum_probs
  end
end


# function performing the random walk
# inputs: current key, current chord degree
# outputs: new key and degree
define :get_next_chord do |k_idx, deg|
  r = rand
  cum_prob = 0
  next_key = nil
  markov[deg].each do |k, v|
    next_key = k
    cum_prob += v
    if cum_prob > r
      break
    end
  end
  # result
  return keys[keys_idx[k_idx] + next_key[0]], next_key[1]
end

# helper for printing notes
define :ntos do |n|
  note_info(n).to_s.split(" ")[1][1..-2]
end


####################
### playing the walk
####################

base = :A2  # first note defining key and octave to start with
ch = [0, :i]  # first chord degree to start with (Amaj7 in this case)
st = 1.6  # overall sleep time between chord progressions

live_loop :walk, auto_cue: false, delay: 0.2 do
  ch = get_next_chord(ch[0], ch[1])
  set :ch, ch
  puts "key: " + ntos(base + ch[0])
  puts "deg: " + ch[1].to_s
  sleep st
end


## bass
live_loop :bass, auto_cue: false do
  sync :ch
  ch = get :ch
  b = base + ch[0]
  deg = ch[1]
  
  my_chord1 = (chord_degree deg, b, :major, 3)
  my_chord1 -= 12 if my_chord1[0] > base + 6
  
  with_synth :fm do
    with_synth_defaults release: 0.1 do
      # basic note
      play my_chord1[0], sustain: st/2
      play my_chord1[0] + 12, sustain: st/2, amp: 0.6 # overtone
      sleep st/2
      # play 5th
      play my_chord1[2] - 12, sustain: st/2
      play my_chord1[2], sustain: st/2, amp: 0.6 # overtone
    end
  end
end


# arpeggio
live_loop :arp, auto_cue: false do
  sync :ch
  ch = get :ch
  b = base + ch[0]
  deg = ch[1]
  
  my_chord2 = (chord_degree ch[1], b, :major, 5)
  my_chord2 = (chord_invert my_chord2, 1) if [:iv, :v].include? deg
  my_chord2 = (chord_invert my_chord2, 2) if deg == :vii
  my_chord2 -= 12 if my_chord2[0] > base + 10
  sca = my_chord2 + my_chord2.reverse[1..3]
  nn = sca.length
  
  with_synth :dpulse do
    with_synth_defaults amp: 0.6, detune: 0.15, sustain: st/nn, release: 0.05, cutoff: 95 do
      sca.each_with_index do |n, i|
        play n
        sleep st/nn if i < nn - 1
      end
    end
  end
end

Using the same idea, my goal is to implement other domains in music theory. The minor keys are the next logical step, but much more difficult to do.

3 Likes

Great piece of work. Well done.

Interesting! That adds a lot more spice than simply using pivot chords and neighbouring keys. It inspires me to do some more experimenting with modulation.

:art: Especially harmonic/melodic minor keys I think. Those shades of minor require special treatment.

I’m interested to see how far those domains can be generalised and unified with your idea, while still making good music.

Agreed. And I expect to need a 3-dimensional random walk. In major, there are the 2 dimensions key+degree and the transition probability depends on degree only. In (melodic) minor keys, I think that the dimensions of the walk will be key+degree+mode, because there is a choice of modes per key+degree pair. Also, the transition probability will depend not only on the current degree, but on the pair degree+mode. And on top, the ‘degrees’ are many more than just 7, because we will need diminished and augmented chords a lot.

Just tried that one:

prop_deg_change = {
  1 => 0.2,
  2 => 0.2,
  3 => 0.1,
  4 => 0.1,
  5 => 0.2,
  6 => 0.1,
}
prop_key_change = {
  -3 => 0.05,
  -2 => 0.1,
  -1 => 0.05,
  1 => 0.0,
  2 => 0.0,
  3 => 0.0,
}

Sounds really dark like a storm coming up. But it’s all in major! What would J.S.Bach have said if you told him that’s music?

That’s a lot of probabilities you’ll have to assign for the ruleset! :thinking: There isn’t an easier way though, given that you’re using a matrix.

Major keys can be very expressive! And there’s voicing too.

I think that functional tonal music wouldn’t surprise him too much? Insofar as it isn’t a totally different paradigm like serialism or sound masses.

The material used in the program is just diatonic chords and relations that are not too far off from what he did. So, my guess is that Bach would have known all that stuff, nothing new. But the questions is whether he dared to make such an extensive use of key changes that are so expressive, as you say. It is not the baroque feeling anymore. He preferred progressions that rather walk along than jump.

Only with the transition matrix I would be confident having covered the space. Linear chains or trees of probabilities produce just a few paths out of infinitely many.

I see, I was thinking of his chromatic pieces, but yes even with those, I get a totally different vibe from baroque harmony.

For sure, the paths are intractable! We try our best to anticipate them with the transition ruleset, leaving room for surprises.

This is my proposal for the Bach feeling:

prop_deg_change = {
  1 => 0.7,
  2 => 0.2,
  3 => 0.0,
  4 => 0.0,
  5 => 0.05,
  6 => 0.0,
}
prop_key_change = {
  -3 => 0.0,
  -2 => 0.0,
  -1 => 0.05,
  1 => 0.0,
  2 => 0.0,
  3 => 0.0,
}

Of course, the bass line would be more sophisticated and not just 1st/5th alternating.

Hmm, trying it out with a few more seeds. I personally am not getting vibes at this stage, but I’m excited to hear where this goes as you add more ideas! Inspired by your approach, I just added common-tone modulation to my project, and I think it sounds much more interesting than just pivot-chord modulation.

At this stage you need quite some fantasy to mentally add a Bach-like bass line. He’s a master of creative bass lines that anticipate the chord changes. Just thinking about modifying the bass loop: get a chord but playing the previous one first, that is, the chord progression loop is running ahead of the voices. Then use the information which chord is to come next and use notes that anticipate the change to come.

I see, so every step would anticipate just the next step. The alternative to that proactive approach is a reactive approach, where the material shifts according to the new key or chord when it happens. My music is entirely reactive. I try to keep it coherent using the repetition of motifs and by minimising voice movement between keys/chords.

In fact, my music is entirely motivic, with no filler material whatsoever. It’s kind of necessary to keep the music coherent. It’s also minimalist.

Well, these are fundamentally different styles creating very different emotions. Building up expectations and fullfilling them (or not), or, just following the flow …

1 Like

Hahaha, yeah, I have a lot to learn from your approach! I’ve kind of ended up making the music that’s “easiest” for me. Which isn’t to say I haven’t put lots of time and effort into my projects. There’s probably an easier way to put it that wouldn’t sound very nice. I’m intellectually lazy or something.

I usually just go with the flow of what’s intuitive to me, but that needs to be moderated with deliberate action sometimes.

In the end however, I like my music, and my taste reflects a bit of me.

1 Like

Do what you like and you will be successful and a lucky man/woman :smiley:

1 Like

Aw, thank you! Yes, let’s all do more of what we like! :blush:

EDIT: Here comes the first second experiment with a bass line that anticipates the chord change.

  1. The chord progression loop informs the voice loops about the current chord as well as the chord to come next
  2. The bass loop calculates the difference between the current and next chord.
  3. Two notes are constructed that serve as a bridge. If possible they are taken from the next chord to come

Here is the complete code so my dear fellows will not have to merge the new bass line into the previous version.

# harmonic random walk in major incl. anticipating bass line
# written by Nechoj

use_debug false
use_random_seed 42

# chord degree sequence
degrees = ring(:i, :iv, :vii, :iii, :vi, :ii, :v)
# prepare hash map to get index in degrees if value is given
degrees_idx = Hash.new
degrees.each_with_index do |k, i|
  degrees_idx[k] = i
end

# key sequence: delta midi notes relative to a base key like :C3
# example when 0=C: C-G-D-A-E-H-Gb-Db-Ab-Eb-B-F
keys = ring(0, 7, 2, 9, 4, 11, 6, 1, 8, 3, 10, 5)
# prepare hash map to get index in keys if value is given
keys_idx = Hash.new
keys.each_with_index do |k, i|
  keys_idx[k] = i
end

# probabilities for changing the index within ring degrees
# prop_deg_change[n]: probability for transition degrees[i] -> degrees[i+n]
prop_deg_change = {
  1 => 0.4,
  2 => 0.1,
  3 => 0.05,
  4 => 0.0,
  5 => 0.1,
  6 => 0.0,
}

# probabilities for changing the index within ring keys
# prop_key_change[n]: probability for transition keys[i] -> keys[i+n]
prop_key_change = {
  -3 => 0.05,
  -2 => 0.05,
  -1 => 0.15,
  1 => 0.05,
  2 => 0.05,
  3 => 0.05,
}

# markovian matrix = map of maps for transition probabilities
markov = Hash.new
degrees.each do |deg|
  markov[deg] = Hash.new
end

# set transition probabilities for key changes
# Interpretation of markov[:i][[-2, :ii]]:
# if last played chord was degree :i and in key keys[n], then
# with probability given by markov[:i][[-2, :ii]]
# change to chord with degree :ii played with key keys[n-2]
markov[:i][[-2, :ii]] = prop_key_change[-2]
markov[:i][[-1, :v]] = prop_key_change[-1]
markov[:i][[1, :iv]] = prop_key_change[1]

markov[:ii][[-3, :vii]] = prop_key_change[-3]
markov[:ii][[-2, :iii]] = prop_key_change[-2]
markov[:ii][[-1, :vi]] = prop_key_change[-1]
markov[:ii][[1, :v]] = prop_key_change[1]

markov[:iii][[-1, :vii]] = prop_key_change[-1]
markov[:iii][[1, :vi]] = prop_key_change[1]
markov[:iii][[2, :ii]] = prop_key_change[2]
markov[:iii][[3, :v]] = prop_key_change[3]

markov[:iv][[-2, :v]] = prop_key_change[-2]
markov[:iv][[-1, :i]] = prop_key_change[-1]
markov[:iv][[1, :vii]] = prop_key_change[1]

markov[:v][[-2, :ii]] = prop_key_change[-2]
markov[:v][[-1, :ii]] = prop_key_change[-1]
markov[:v][[-1, :v]] = prop_key_change[-1]

markov[:vi][[-2, :vii]] = prop_key_change[-2]
markov[:vi][[-1, :iii]] = prop_key_change[-1]
markov[:vi][[1, :ii]] = prop_key_change[1]
markov[:vi][[2, :v]] = prop_key_change[2]

markov[:vii][[-1, :iv]] = prop_key_change[-1]
markov[:vii][[-1, :vii]] = prop_key_change[-1]
markov[:vii][[3, :v]] = prop_key_change[3]

# set transition probabilities for chord changes without key change
degrees.each do |deg|
  this_idx = degrees_idx[deg]
  range(1, 7, 1).each do |i|
    markov[deg][[0, degrees[this_idx + i]]] = prop_deg_change[i.to_i]
  end
end

# finally set probabiltities for no change (play same chord again)
# adjust row sum of markov map equal to 1.0
degrees.each do |deg|
  cum_probs = 0.0
  markov[deg].each_value do |v|
    cum_probs += v
  end
  if cum_probs > 1
    puts "probabilties greater than 1; normalizing", deg
    markov[deg][[0, deg]] = 0.0
    markov[deg].each_key do |k|
      markov[deg][k] /= cum_probs
    end
  else
    markov[deg][[0, deg]] = 1.0 - cum_probs
  end
end


# function performing the random walk
# inputs: current key, current chord degree
# outputs: new key and degree
define :get_next_chord do |k_idx, deg|
  r = rand
  cum_prob = 0
  next_key = nil
  markov[deg].each do |k, v|
    next_key = k
    cum_prob += v
    if cum_prob > r
      break
    end
  end
  # result
  return keys[keys_idx[k_idx] + next_key[0]], next_key[1]
end

# helper for printing notes
define :ntos do |n|
  note_info(n).to_s.split(" ")[1][1..-2]
end

# helper to calculate difference between 2 scales
# returns all notes in sca2 that are not contained in sca1, but in bsca
# octaves do not matter, just degree
define :scale_diff do |sca1, sca2, bsca|
  bsca = bsca.to_a
  diff = Array.new
  sca2.each do |n2|
    next unless bsca.include? n2
    found = false
    sca1.each do |n1|
      if n1%12 == n2%12
        found = true
        break
      end
    end
    diff.push(n2) unless found
  end
  return diff
end

####################
### playing the walk
####################

base = :A2  # first note defining key and octave to start with
ch = [0, :i]  # first chord degree to start with (Amaj7 in this case)
st = 1.6  # overall sleep time between chord progressions

live_loop :walk, auto_cue: false, delay: 0.2 do
  # get next chord and inform voices
  ch_next = get_next_chord(ch[0], ch[1])
  set :ch_next, ch_next
  # tell voices actual chord to play now
  set :ch, ch
  puts "key: " + ntos(base + ch[0])
  puts "deg: " + ch[1].to_s
  sleep st
  ch = ch_next  # assignment will not work before sleep as ch has been set as cue
end


## bass
live_loop :bass, auto_cue: false do
  sync :ch
  # current chord
  ch = get :ch
  b = base + ch[0]
  deg = ch[1]
  # chord to come next
  ch_next = get :ch_next
  b_next = base + ch_next[0]
  deg_next = ch_next[1]
  
  # construct notes from current and next chord
  chord_now = (chord_degree deg, b, :major, 4)
  scale_now = (scale, b, :ionian, num_octaves: 3)
  scale_now -= 12
  chord_next = (chord_degree deg_next, b_next, :major, 4)
  
  # build bridge notes leading to next chord
  # start with difference notes
  bridge = scale_diff(chord_now, chord_next, scale_now)
  # fill up with 5th and 3rd of next chord if necessary
  if bridge.length == 0
    # add 3rd if possible
    if scale_now.to_a.include? chord_next[1]
      puts "adding 3rd"
      bridge.push(chord_next[1])
    else
      bridge.push(chord_now[1])
    end
  end
  
  if bridge.length == 1
    # add 5th if possible
    if scale_now.to_a.include? chord_next[2]
      puts "adding 5th"
      bridge.push(chord_next[2])
    else
      bridge.push(chord_now[0])
    end
  end
  
  # reverse makes bass line falling
  bridge = bridge.reverse
  
  bridge.each do |n|
    puts "anticipating " + ntos(n)
  end
  
  nb = bridge.length
  stb = st/(2*nb)  # sleep time for bridge notes
  
  with_synth :fm do
    with_synth_defaults release: 0.1 do
      # basic note
      n1 = chord_now[0]
      n1 -= 12 if n1 > base + 6
      n1 -= 12 if n1 > base + 6
      play n1, sustain: st/2-0.1
      play n1 + 12, sustain: st/2-0.1, amp: 0.6 # overtone
      sleep st/2
      
      bridge.each_with_index do |n, i|
        n -= 12 if n > base + 8
        n -= 12 if n > base + 8
        play n, sustain: stb-0.1
        play n + 12, sustain: stb-0.1, amp: 0.6 # overtone
        sleep stb if i < nb-1
      end
    end
  end
end

# arpeggio
with_fx :reverb,room: 0.8, mix: 0.3 do
  live_loop :arp, auto_cue: false do
    sync :ch
    ch = get :ch
    b = base + ch[0]
    deg = ch[1]
    
    chord_now = (chord_degree ch[1], b, :major, 5)
    chord_now = (chord_invert chord_now, (knit 0, 3, 1, 2, 2, 1).choose)
    chord_now -= 12 if chord_now[0] > base + 10
    sca = chord_now + chord_now.reverse[1..3]
    nn = sca.length
    
    with_synth :dtri do
      with_synth_defaults amp: 0.8, attack: 0.05, sustain: st/nn-0.1, release: 0.08, cutoff: 100 do
        sca.each_with_index do |n, i|
          play n+12
          sleep st/nn if i < nn-1
        end
      end
    end
  end
end

EDIT: Update on the bridge notes for the bass line. So far I am quite happy with the result. The bridge notes almost perfectly anticipate the next chord to come.
EDIT2: Replaced :dpulse by :dtri. Sound is more pleasant and better hamonizing with :fm bass.

2 Likes

Awesome.
I wonder if this could be reimplemented without using a pre-defined markovian heuristic or domain specific knowledge about keys, etc, but rather defining a fitness function that scores modulations or chord changes based on how likely they are to sound good (based on note distance and kind of intervals?) and use it to implement a more free-form genetic algorithm. Would make stranger music, I think, but it might sound OK.

I have some thoughts in the domain of free-form genetic algorithms, because I’ve been working with note distance and intervals for the past 3 months. My work is polyphonic rather than homophonic, so I approach chords with voices. Only recently have I begun to think about Markov matrices.

I don’t have a fitness function for you, but with regards to chords, I think that every progression has its place, and that voicing can have a greater impact than the chord selected. So that leads us to effective voicing. In that regard, the general melodic guideline that has created my favourite works so far is to limit melodic intervals (total or stepwise) to anything between a sixth and an octave. That’s about it.

I can show you some experiments I’ve made if you’re interested :slight_smile:

Oh, definitely, please do.