Openai´s chatbot coding Sonic Pi

Creations & Ideas
1

I ckecked out the chatbot at
https://chat.openai.com/chat

Asked it to do some coding examples in Sonic Pi.
This is the result for generative music:

use_bpm 120

Set the root note of the scale

root = :c4

Choose a random scale mode

mode = [:ionian, :dorian, :phrygian, :lydian, :mixolydian, :aeolian, :locrian].choose

Generate a scale using the root note and mode

music_scale = scale(root, mode)

Set the number of notes to play

num_notes = 32

Choose a random synth

synth = [:sine, :square, :tri, :saw].choose

Set the release time for the notes

release = 0.5

Set the cutoff frequency for the synth

cutoff = rrand(60, 120)

Set the sleep time between notes

sleep_time = 0.125

Set the number of times to loop

num_loops = 4

Play the music indefinitely

loop do
num_loops.times do
# Play the notes of the scale
num_notes.times do
use_synth synth
play music_scale.choose, release: release, cutoff: cutoff
sleep sleep_time
end

# Choose new options for the next loop
root = music_scale.choose
mode = [:ionian, :dorian, :phrygian, :lydian, :mixolydian, :aeolian, :locrian].choose
music_scale = scale(root, mode)
synth = [:sine, :square, :tri, :saw].choose
release = rrand(0.1, 1)
cutoff = rrand(60, 120)
sleep_time = rrand(0.1, 0.5)

end
end

I also tried to get it to code a fugue. After some nudging, this is what it came up with:

G Major Scale

g_major_scale = [:g3, :a3, :b3, :c4, :d4, :e4, :fis4]

Subject

subject = [:g3, :g3, :g3, :b3, :d4]

Voice 1

in_thread do
use_synth :saw
play_pattern_timed subject, [0.5, 0.5, 0.5, 1, 1]
sleep 1
play_pattern_timed subject, [0.5, 0.5, 0.5, 1, 1]
end

Voice 2

sleep 0.5
in_thread do
use_synth :square
play_pattern_timed subject, [0.5, 0.5, 0.5, 1, 1]
sleep 1
play_pattern_timed subject, [0.5, 0.5, 0.5, 1, 1]
end

Voice 3

sleep 1
in_thread do
use_synth :tri
transposed_subject = subject.map { |note| g_major_scale[(g_major_scale.index(note) + 7) % 7] }
play_pattern_timed transposed_subject, [0.5, 0.5, 0.5, 1, 1]
sleep 1
play_pattern_timed transposed_subject, [0.5, 0.5, 0.5, 1, 1]
end

Voice 4

sleep 1.5
in_thread do
use_synth :saw
transposed_subject = subject.map { |note| g_major_scale[(g_major_scale.index(note) + 12) % 7] }
play_pattern_timed transposed_subject, [0.5, 0.5, 0.5, 1, 1]
sleep 1
play_pattern_timed transposed_subject, [0.5, 0.5, 0.5, 1, 1]
end

Both examples are not impressive in itself, but I suppose in not much time we will be directing a bot to code stuff for us?
Has anyone else tried this?

3 Likes
2

In a sense, it’s a much easier job to train a model to generate code which reproduces a particular recorded sample, since one can generate infinite training data for that problem. It seems more difficult to have sufficient training data for generating code from a sophisticated textual description, like “play piano like Hania Rani”. It’s still an interesting possibility, and I suspect it’d be possible to link up DL models to drive synthdef controls or harmonic motion.

disclaimer, my work is related to ML techniques for nonlinear dynamical systems, so I see music through that lens

1 Like
3

also I meant to reply to this point: you can look at GitHub Copilot to see what this looks like: you write a function signature and a docstring and it can handle boilerplate coding tasks, but it’s not really up to complex stuff.

2 Likes
4

What tools would you use to work from a recorded sample? I have a music background and only dabble with coding. ChatGPT only offers text input and output, I did not try any other options, but I´m curious.

5

To train a network, we’d need some examples, each of which would consist of a piece of code and the resulting waveform. A model could be set up that either reads the code and predicts the waveform or vice verse, and such a model can be trained on the examples. Once the training is done, then the resulting network can be used for inference.

With inference, one would input a waveform and the model would generate some code (or vice versa, but we already have synth engines to generate waveforms from code). This could be done with almost any tool, though if I were trying to do this, I look for which ML toolbox has some add ons for audio processing. For instance Hugging Face has 57 model results for “music” and the first one generates sheet music from textual description:

Making all that practical on a RPi is another story: the model size is 600 MB (though recent techniques for model reduction might help), but the inference step is cheap and could be set up as an internet API anyone can use and I don’t see latency for that step being a big deal: writing code by hand has a high latency :wink:

edit I see this model in particular generates melodies for different styles in ABC notation (?) and could be directly used in SP or convert-able to SP code perhaps.

1 Like
6

the paper behind that particular model is on arxiv

and has some interesting bits about the dataset they used,

To make models learn the relationship between natural language and symbolic music, we collected as many text- music pairs as possible, composing a text-music dataset called Textune. This dataset contains 282,870 English text- tune pairs, where all tunes are represented in ABC no- tation3. As ABC notation encodes music scores into se- quences of ASCII characters, using this music notation sys- tem makes it easy to model the text-to-music generation task as a sequence-to-sequence problem.
… In general, the valid text descriptions in Textune can be categorised as follows: 1) musical anal- ysis (e.g., tonal analysis and harmonic analysis), 2) meta- information (e.g., key and meter), 3) the context in which the piece was composed (e.g., history and story), and 4) subjec- tive perceptions (e.g., sentiment and preference)

If the dataset is available, then it could be potentially retrained jointly with code samples, which would allow directly inferring code sample corresponding to textual description. One limitation would be that this wouldn’t really enable exploring aspects of music which aren’t captured by those data, such as timbre. But it sure would be a neat start!

2 Likes
7

Yeah, I just tried this a few months back too. I was trying to get it to write code for loop chopping and rearranging for breakbeats but didn’t have much luck with it. Also got it to write a melody but it was pretty barebones.

I think it will get there though. I have used it for some non-coding things like writing personas, designing characters, rewriting/condensing texts and stuff like that with a lot of success. I also tried their MIDI AI and it’s pretty decent at writing MIDI for songs.

2 Likes