Playing Midi Files on Sonic Pi

Creations & Ideas
1

Playing-Midi-Files-on-Sonic-PI

Playing Midi Files on Sonic PI

This project starts by using a Midi to CSV program, downloaded from the Web, to convert a Midi format file into CSV format. The project code then operates on the CSV file to produce a CSV for each Midi track. These files contain two columns containing Midi notes, and a duration time. An example program uses these files to play the notes using Sonic PI.

Discussion

As a first cut I’m pleased with the results, although the playback has some synchronization problems. This is related to the paradigm shift of timing in Midi files and the way Sonic Pi tracks the time. Other parameters in the Midi file have been ignored such as the velocity values.
The purpose of this posting is to record what has been done so that the experts in Sonic Pi can make suggestions.

InThread Group

1 Like
2

This looks interesting. I have used and developed a bit a processing script by Hiroshi Tachibana @tcbnhrs for about 18 months to do this. It has limitations, only working on a single voice at a time, but I have successfully used it to convert some quite large midi pieces for use on Sonic PI. see here
Your technique using csv files looks as if it could potentially be easier, so I look forward to studying it.

1 Like
3

Fixed the sync problem. It were the unequal dummy sleeps in the live_loops

Check this one out, 8 tracks, 6 voices The Art of the Fugue JS Bach

TAOTF

Adele Polka

1 Like
4

The Bach sounds great!

5

I had a go at using your softeware. I donwloaded and built the midicsv program on my Mac and then converted a file successfully to csv format.
However, I couldn’t get your converter program to work. It only get a list of rest entries in the voice csv files lik

:r	0.576
:r	0.192
:r	0.384
:r	0.384
:r	0.576
:r	0.192
:r	0.384
:r	0.576

Looking at the csv file produce by midicsv the major diference I noticed is that the file did not have any Note_off_c messages but instead just used Note_on_C with velocity 0, as do I think many other midi files. The example you used had both
Note_on_c and Note_off_c messages.
Is this likely to be the problem?
The beginning of the csv file I used is below

0, 0, Header, 1, 6, 384
1, 0, Start_track
1, 0, Title_t, "control track"
1, 0, Text_t, "creator: "
1, 0, Text_t, "GNU LilyPond 2.18.2           "
1, 0, Time_signature, 4, 2, 18, 8
1, 0, Tempo, 600000
1, 0, End_track
2, 0, Start_track
2, 0, Title_t, "trackB:voiceA"
2, 0, Control_c, 0, 7, 100
2, 0, Control_c, 0, 7, 100
2, 0, Control_c, 0, 7, 100
2, 0, Key_signature, -1, "major"
2, 384, Note_on_c, 0, 72, 90
2, 768, Note_on_c, 0, 72, 0
2, 768, Note_on_c, 0, 72, 90
2, 1152, Note_on_c, 0, 72, 0
2, 1152, Note_on_c, 0, 74, 90
2, 1536, Note_on_c, 0, 74, 0
2, 1536, Note_on_c, 0, 76, 90
2, 2688, Note_on_c, 0, 76, 0
2, 2688, Note_on_c, 0, 77, 90
2, 3072, Note_on_c, 0, 77, 0
2, 3072, Note_on_c, 0, 74, 90
2, 3648, Note_on_c, 0, 74, 0
2, 3648, Note_on_c, 0, 72, 90
2, 3840, Note_on_c, 0, 72, 0
2, 3840, Note_on_c, 0, 70, 90
2, 4224, Note_on_c, 0, 70, 0
6

To debug insert a sleep so that the XReadMidiCsv4a.rb program will show the progress.

    lineno+=1
    sleep 0.2

Your header specifies type 1 Midi as did the Polka file.
I would think it would be different. Puzzling?

        if row[type]==" Note_on_c"
          dt=(row[time].to_i-tstart)/1000.0
       if dt >0.00999
         sss=[:r,dt]                                                  # Here is where the rests are processed
         voices[trackn]=voices[trackn].push(sss) 
        end #dt
        tstart=row[time].to_i
        channel=row[3].to_i
        tone=row[4].to_i
        velocity=row[5].to_i
        sss= [tone,tstart.to_f]
        nteon=nteon.push(sss)
      end # note on

And this is where the note is inserted, and it could be a chord.

      if row[type]==" Note_off_c"
        tend=row[time].to_i
        channel1=row[3].to_i
        tone1=row[4].to_i
        velocity1=row[5].to_i
        ttt=nteon.slice!(findit(tone1,nteon))
          voices[trackn]=voices[trackn].push([tone1,(tend-ttt[1])/1000.0])
        end # note off

I wonder how your file handles a rest and chords without using the note off?

7

This looks like a possible solution:
Midi Tutorial

There is a special case if the velocity is set to zero. The NOTE ONmessage then has the same meaning as a NOTE OFF message, switching the note off.
From the sample you provided above:

2, 384, Note_on_c, 0, 72, 90
2, 768, Note_on_c, 0, 72, 0    # really a note off as the last paramter is a zero velocity.

Channel Events

These events are the “meat and potatoes” of MIDI files: the actual notes and modifiers that command the instruments to play the music. Each has a MIDI channel number as its first argument, followed by event-specific parameters. To permit programs which process CSV files to easily distinguish them from meta-events, names of channel events all have a suffix of “_c”.

From Midi to CSV site:

Midi to CSV

Track, Time, Note_on_c, Channel, Note, Velocity
Send a command to play the specified Note (Middle C is defined as Note number 60; all other notes are relative in the MIDI specification, but most instruments conform to the well-tempered scale) on the given Channel with Velocity (0 to 127). A Note_on_c event with Velocity zero is equivalent to a Note_off_c.
Track, Time, Note_off_c, Channel, Note, Velocity
Stop playing the specified Note on the given Channel. The Velocity should be zero, but you never know what you’ll find in a MIDI file.

I need to read the fine manual!

8

Yes that is what the midi file is doing it is using Note_on_c with a zero velocity to turn the note off. I’m not sure if your software allows for this. I haven’t had time to study it fully yet. My comments came after a first run, using a midi source file which plays on a midi player correctly, and converts to Sonic-Pi format OK using my processing script.

I’ll try adding the debugging stuff and investigate further, probably tomorrow.

9

Here is a fix to try. I would do it but I don’t have your midicsv file to test it with.
I looks for note on command with a zero velocity and changes it to a note off before it reaches the part that parses note on and note off.

while loopflag>0
  if loopflag>0
    text1= text.split("\n",2)
    ss1=text1.slice(0)
    text=text1[1]
    row=ss1.split(",")
    puts row
    puts lineno,row
    lineno+=1
    #    sleep 0.2 # uncomment for debugging
   # Insert this code
   if row[type]==" Note_on_c"
   if (row[5].to_i) == 0  # then it's really a note_off
     row[type]= " Note_off_c"
 end # if row[5]
 end # if row[type]
   #end insertion
    if row[type]!=" End_of_file"
10

Thanks for that. I fad the file through a regex search and replace and did something similar last night but haven’t tried the result yet. However it would be better to do it all in the one as you have here.
If you want to try out from my midi file you can get it here

11

It works perfectly now. Thanks a lot.

(I inadvertenly edited your patch above (as I have admin rights). I think it is back as it was now Apologies.

12

Gihub Link

See the BugFix1 section, that addresses the problem and the fix.

13

I’ve been working on automating the PlayMidiCsva.rb file, and have now got it so that it can handle any number of voice csv files voiceA0.csv up to voiceA15.csv automatically
without you having to do anything to the file. It detects if they are present and if so whether they are empty and adjusts accordingly. By changing one line, you can allocate any synth either to all or to specified parts. You can even alter them whilst it is running!

I’ll just do a bit more testing and I’ll post you a copy.

14

I’ve now tested my automated version on a 9 part fugue that lasts over 20 mins and it handled it perfectly.
The one thing I don’t understand is the way the live_loop plays the commands stored in the array.
for example

i1=0
live_loop :LL1 do
  use_synth :piano
  #puts voice2[i1]
  puts midi2note(voice2[i1][0].to_i),voice2[i1][1]
  if voice2[i1][0]!= ":r"
    play voice2[i1][0].to_i,sustain: voice2[i1][1]
    sleep 0.000001
  else
    sleep voice2[i1][1]
  end
  i1+=1
  if i1>=voice2.length
    stop
  end
end

This does work, but I would have expected code like this:

i1=0
live_loop :LL1 do
  use_synth :piano
  #puts voice2[i1]
  puts midi2note(voice2[i1][0].to_i),voice2[i1][1]
  if voice2[i1][0]!= ":r"
    play voice2[i1][0].to_i,sustain: voice2[i1][1]
  end
    sleep voice2[i1][1]
  end
  i1+=1
  if i1>=voice2.length
    stop
  end
end

as play normally executes taking up zerotime in the timeline, and it is usually followed by a sleep equal to the duration of the note required.
When dealing with a rest you merely omit the play and proceed to the sleep.
But in your code you don’t have any allowed sleep following the note play apart from a very very small one which keeps the live_loop from complaining there is no sleep in the loop.
You only have the duration sleep present in the case of the rest.
However the code I would have expected shown above does not work here, although it DOES work in the test example below

notes=scale(:c4,:major,).to_a+[:r]*5+scale(:c4,:major).reverse.to_a
durations=[0.2]*notes.length
i=0
live_loop :test do
  if notes[i]!=:r
    play notes[i],sustain: durations[i]
  end
  sleep durations[i]
  i+=1
  stop if i>=notes.length
end

The only thing I can think is the the use of the file handling upsets Sonic Pi in some way that alters the way the play command responds.

15

Here is the link to my automatic player file playmidiCsvaAuto.rb
Interested to see how you find it.

16

Hi @robin.newman thanks for the Auto software, I’ll give it a go tomorrow.

As for the file structures usage of sleep, here are some cases to consider.

From Voice3.csv of the Adele Polka

C2	0.119
:r	0.24
C3	0.119
E3	0.119
G3	0.119
:r	0.24
C2	0.119
:r	0.24
C3	0.119
E3	0.119
G3	0.119
:r	0.24

Notice the alternating single notes and chords.

From Silverswan2.csv

:r	0.384	0
72	0.384	90
:r	0.384	0
72	0.384	90
:r	0.384	0
74	0.384	90
:r	0.384	0
76	1.152	90
:r	1.152	0
77	0.384	90
:r	0.384	0

This is a string of single notes no chords. The extra column is the Start note velocity which can be used to change the amplitude.

live_loop :LL1 do
  use_synth :piano
  #puts voice2[i1]
  puts midi2note(voice2[i1][0].to_i),voice2[i1][1]
  if voice2[i1][0]!= ":r"
    play voice2[i1][0].to_i,sustain: voice2[i1][1]
    sleep 0.000001
  else
    sleep voice2[i1][1]
  end
  i1+=1
  if i1>=voice2.length
    stop
  end
end

The note duration may not be as long as the duration between notes. There may be a rest that Midi doesn’t specifically notates as a separate command. These rests may be in the musical score or just because a finger was fast of slow on the keyboard.

Consider coding this from Midi:

use_synth :fm
a=chord(:c4,:major)
puts a

play a
sleep 0.25
sleep 0.25

play a[0],sustain:0.5
play a[1],sustain:0.125
play a[2],sustain:0.25
sleep 0.75

play chord(:d4,:major)
sleep 0.25

On the treatment of the “:r”.
The notes are numbers and the “:r” should be a nil. ( I seem to have trouble with this a lot)

[
[:C4,0.25],
[nil,0.25],
[nil,0.25],
[:C4,0.5],
[:E4,0.125],
[:G4,0.25],
[nil,0.75],
#play chord(:d4,:major)
[[:d4,:fs4,:A4],0.25]
[nil,0.25],

There seem to be two basic apporaches to chords: Code them into one voice or split them into separate tracks.

17

Yes, I had a think about this overnight and it came to me this morning just before I got up that it was probably to do with the way of playing chords. You could send several notes with durations, but ignore any sleep (apart from a tiny one to keep the live_loop happy and then send a sleep at the end to keep the timing of the playing correct.
I realised the example I send only dealt with monophonic input.
Your examples above make complete sense, and it is nice solution that you use.

18

Hi @robin.newman

I gave your Auto program a try, but no sound.
Possibilities:
I’m on a PC which doesn’t have the latest SP that you have on your Mac.
The files I used had the velocity added as a 3ird column. I’ll try again later today with some earlier files.

Some things to consider:
The first program takes the MidiCSV and creates the arrays and then writes them to files.
I did this because with the sleep in the code for debugging, it was taking a long time to process the input and of course having the output in files helped the debugging process.
It would be possible to add your auto code to the amended program and eliminate the output files, if the debugging sleep is removed.

I’ve inserted sleep into some of the header sections so they appear in the right window.

 ├─ Timing warning: running slightly behind...
 ├─ 5 ["1", " 0", " Key_signature", " -1", " \"minor\""]
 ├─ 6 ["1", " 0", " Tempo", " 1000000"]
 └─ "tempo1=" 1000.0
 
{run: 2, time: 0.4}
 ├─ 7 ["1", " 0", " Tempo", " 1500000"]
 └─ "tempo1=" 1500.0
 
{run: 2, time: 0.6}
 ├─ 8 ["1", " 4351", " Tempo", " 333333"]
 └─ "tempo1=" 333.333
 
{run: 2, time: 0.8}
 ├─ 9 ["1", " 4352", " Tempo", " 600000"]
 └─ "tempo1=" 600.0
 
{run: 2, time: 1.0}
 ├─ 10 ["1", " 12992", " Tempo", " 750000"]
 └─ "tempo1=" 750.0

There is more information buried in the Midi files that needs attention.

For your Auto program, changing the Synth used may need additional treatment to produce good quality.
Your code is amazing :slight_smile:

19

Really enjoying playing with this. I think it has great potential, and as you say there are other bits of the midi file that might be pulled out and incorporated. I have come across one or two minor issues. Non utf-8 characters in some of the header bits, easily stripped out with a text editor. One piece I have just worked on got funny timings at the end (multiple tempo changes) and one note left hanging. Again edited the .csv file to correct.
I like the idea of writing the files out and will take a look at that.
My latest play has been with Widor Tocatta, which I have voiced for three different synths, added with_fx :level for each part to balance them, and also added verb. I also altered the play command to use the full sustain value, but added a 0.2 times that value for the release which seems to work well for non percussive (eg piano, pluck) synths.
I"ve posted the resulting audio here what do you think.

20

Hi @robin.newman,

I think you made the case for intermediate files. I like your rendition of the Widor Tocatta.

I’ve been busy with other life tasks and a little side project.

Espruino Project

A few bars into Bach’s Toccata an Fugue in D minor, there is a pyramid chord that presents a problem when translating from Midi to SP. I need to look at it further to see if SP can even do it in one live-loop.

I’m thinking of a new command for SP, call it Nod, It’s like sleep but doesn’t turn off the sound.
Play :D4
Nod 0.25
Play :Ds5
sleep 0.25

Is Nod a possibility?