MIDI note on's and note off's sometimes misordered

Support, Help & Resources
1

Here’s a toy example that reproduces the problem I’m having with much longer functions:

use_bpm 60

define :myFunction do
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 1
  midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
end

live_loop :myLoop do
  myFunction
end

MIDI Monitor reveals this plays back, at one point, as follows:

16.369	From Bus 1	Note On	1	C3	100
17.369	From Bus 1	Note Off	1	C3	127

17.370	From Bus 1	Note On	1	C3	100
18.369	From Bus 1	Note Off	1	C3	127

18.369	From Bus 1	Note On	1	C3	100
19.369	From Bus 1	Note On	1	C3	100
19.370	From Bus 1	Note Off	1	C3	127
20.369	From Bus 1	Note Off	1	C3	127

20.369	From Bus 1	Note On	1	C3	100
21.369	From Bus 1	Note Off	1	C3	127

Those first two Note On’s get followed by their Note Off’s as expected, but the third Note On has the fourth one sneak in right before its Note Off. As a result, the fourth Note On gets clipped because it gets offed by the other note’s Note Off. Somehow, that fourth Note On gets executed before the third Note Off, messing things up.

In my real code, this problem happens about 10% of the time, a Note On from a subsequent loop getting executed before the Note Off from a preceding loop. There’s no discernible pattern as to when it will happen. This kludge fixes the error:

live_loop :myLoop do
  myFunction
  sleep 0.001
end

but I’d like to understand why this happens, and if there’s a way to avoid this weird execution misordering without making a radical change to the way I’m doing things here. As I said, it’s a toy, and the larger project depends on working this way, with all sleep’s in the functions and none in the loops.

2

I guess I can simply do something like:

NOTE_OFF_ALLOWANCE = 0.01

define :myFunction do
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 1 - NOTE_OFF_ALLOWANCE
  midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1

  sleep NOTE_OFF_ALLOWANCE
end

That’s fine, I guess, because it hides the mess inside the functions and leaves the live loops clean.

1 Like
3

If you use the midi command which uses the sustain: opt to determine how long a note is on for then it seems to work OK.
eg (my port name was different)

use_bpm 60

define :myFunction do
  midi 60, 100, sustain: 1, port: "iac_driver_sonicpi", channel: 1
  sleep 1
end

live_loop :myLoop do
  myFunction
end

The midi monitor output is:

19:38:10.198	From sonicpi	Note On	1	C3	100
19:38:11.189	From sonicpi	Note Off	1	C3	127
19:38:11.199	From sonicpi	Note On	1	C3	100
19:38:12.188	From sonicpi	Note Off	1	C3	127
19:38:12.198	From sonicpi	Note On	1	C3	100
19:38:13.189	From sonicpi	Note Off	1	C3	127
19:38:13.199	From sonicpi	Note On	1	C3	100
19:38:14.188	From sonicpi	Note Off	1	C3	127
19:38:14.198	From sonicpi	Note On	1	C3	100
19:38:15.189	From sonicpi	Note Off	1	C3	127
19:38:15.198	From sonicpi	Note On	1	C3	100
19:38:16.189	From sonicpi	Note Off	1	C3	127
19:38:16.198	From sonicpi	Note On	1	C3	100

Here Sonic Pi works out the scheduling works as you might expect. With separate events generated by using midi_on and midi_off separately there is effectively a race condition as you are simultaneously triggering two midi events at the same time.

@samaaron may be able to explain it better. BTW the midi subsystem is revamped in 3.4dev and is still undergoing further possible revision.

1 Like
4

I still get the same problem with the sustain: opt on my non-toy code. It’s less frequent than before, but MIDI Monitor still shows occasional breaks in the Note Off/Note On pattern so that a sneaky new note gets offed by the previous cycle’s Note Off.

ADDED: sustain: actually is a fix. The continued fails were due to other routines getting called, not the one with the sustain: fix, after implementing sustain: everywhere, it worked.

Even though sustain: works, it clobbers a timing variation option I’ve implemented that requires dealing with Note On and Note Off separately, so I also tried using mutex’s in different places. Nothing changed because the new iteration of the live loop just gets in there before the last one’s completed, steals the semaphore, and does a Note On that immediately gets snuffed by the previous iteration’s Note Off. Maybe there’s a clever use of mutex’s that will work?

This is a real bugaboo. I’ve been getting these snuffed notes for months, but only with VST plugins, not hardware (I think). Because I believe I don’t have this problem with hardware MIDI synths on my mioXL hub, I’ve been assuming that there was something going on with the plugins’ settings that I could look into later. Also, randomly cutting notes short often doesn’t sound bad, depending on what you’re trying to do, so I’ve written a lot of code without first solving a fundamental problem. At this point, though, I don’t actually understand how a subsequent instance of a loop can begin before the previous one has finished, so a more detailed explanation would be useful.

In the meantime, I’ll find the time to verify behavior with external MIDI synths, which I don’t spend as much time with due to the convenience of going totally portable with software synths.

ADDED: I do get the same problem when sequencing external hardware.

If there are other suggestions of things to try, I’m interested. Do you see any problems with my NOTE_OFF_ALLOWANCE kludge? A big drawback is that it fills the log with “running behind” warnings, and I wonder if small timing errors can accumulate to throw simultaneous live loops audibly out of sync.

I wonder if things work differently on Windows?

5

Had another brief play. Running the function in real time seems to work.

use_bpm 60

define :myFunction do
  use_real_time
  midi_note_on 60, 100, port: "iac_driver_sonicpi", channel: 1
  sleep 1
  midi_note_off 60, 127, port: "iac_driver_sonicpi", channel: 1
end

live_loop :myLoop do
  myFunction
end

08:32:29.028	From sonicpi	Note On	1	C3	100
08:32:30.027	From sonicpi	Note Off	1	C3	127
08:32:30.027	From sonicpi	Note On	1	C3	100
08:32:31.026	From sonicpi	Note Off	1	C3	127
08:32:31.027	From sonicpi	Note On	1	C3	100
08:32:32.027	From sonicpi	Note Off	1	C3	127
08:32:32.028	From sonicpi	Note On	1	C3	100
08:32:33.029	From sonicpi	Note Off	1	C3	127
08:32:33.029	From sonicpi	Note On	1	C3	100
08:32:34.027	From sonicpi	Note Off	1	C3	127
08:32:34.027	From sonicpi	Note On	1	C3	100
08:32:35.026	From sonicpi	Note Off	1	C3	127
08:32:35.027	From sonicpi	Note On	1	C3	100
08:32:36.028	From sonicpi	Note Off	1	C3	127
08:32:36.029	From sonicpi	Note On	1	C3	100
08:32:37.027	From sonicpi	Note Off	1	C3	127
08:32:37.027	From sonicpi	Note On	1	C3	100
08:32:38.026	From sonicpi	Note Off	1	C3	127
1 Like
6

In the three minutes since you posted, I was editing my post above as follows:

ADDED: sustain: actually is a fix. The continued fails were due to other routines sometimes getting called, not the one with the sustain: fix. After implementing sustain: everywhere, it worked.

ADDED: I do get the same problem when sequencing external hardware.

I’ll also try use_real_time. Are there any negatives to that? I mean, why don’t we just always use real time as a general principle?

7

Real time can help. Can b e a problem with a very processing intensive piece, or a slower processor, but generally I use it quite a lot. I tend to use it more with pieces involving lots of OSC calls. Better to have it opt in, rather than opt out I think.
EDIT I checked these will a real external synth. The original IS a problem. sustain and realtime seem OK.

8

So if I just put a use_real_time at the top of my buffer (not per loop) and forget about it, that’s a viable way to go?

By the way, is using sustain: supposed to present any particular advantages over explicit Note On’s and Note Off’s aside from my example? It does simplify code, and the affect it had on my timing variation didn’t make it work badly, just differently.

9

Hi there,

to help understand what’s going on here it might help to slightly rework this code:

The following is essentially the same code with the same behaviour:

use_bpm 60
live_loop :my_loop do
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 1
  midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
end

(All I did was remove the function definition and call and bring the code inline into the live loop).

Now, if we unroll the live loop, we can see that it’s doing the following:

  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 1
  midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
  # loop round...
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 1
  midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
  # loop round...
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 1
  midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
  # etc...

Now we can see that note_off happens at the same time as the first note_on as there’s no call to sleep between them. The current implantation of the MIDI scheduler orders events by time, and as both of these events happen at the same time for the same note, there’s a so-called race condition and it will be random which one “wins” i.e. gets converted to a MIDI message and sent to the device first.

To fix this, you need to ensure that your note_off happens before the next note_on for a given note. You could therefore do something like:

live_loop :my_loop do
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 0.9
  midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
  sleep 0.1
end

Of course, this means you now have to worry about adding all the sleep values together to ensure they’re the right value (1 in this case). An alternative is to use at or time_warp to jump to the right time for a specific block, perform the action and then revert time back to what it was. See the documentation for at or time_warp for more details. However, this is essentially what the midi fn does under the scenes:

live_loop :my_loop do
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  time_warp 0.9 do
    midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
  end
  sleep 1
end

Hope that this helps!

1 Like
10

Yes, I now understand that the next loop begins at the exact same time as the previous one’s ending. I guess the use_real_time prevents that exactness. I’m now doing

define :myFunction do
  midi_note_on 60, 100, port: "iac_driver_bus_1", channel: 1
  sleep 1

  time_warp -0.001 do
    midi_note_off 60, 127, port: "iac_driver_bus_1", channel: 1
  end
end

which looks pretty much the same as the NOTE_OFF_ALLOWANCE arithmetic I did above, but which leads to tons of complaints in the log. time_warp works fine even with the small increment, which seems to me desirable to reduce the likelihood of messing with other timing modifications I’d like to be able to make.

Is there any reason to prefer one or the other of use_real_time and time_warp (and perhaps sustain:), or is it pretty much a toss-up given a powerful enough CPU? time_warp has the advantage of employing a explicitly intended functionality rather than benefiting from a side-effect, but maybe that’s a difference that makes no difference.

11

This thread saved me some trouble, thank you.

Just noting that sustain with VST instruments didn’t work for me, some of them never turned off. In my function definitions I’ve set a parameter equivalent to your NOTE_OFF_ALLOWANCE with a default of 0 - the misordering only matters when repeating the same note.

12

Do you get “falling behind” warnings in the log when you do the NOTE_OFF_ALLOWANCE method? I was getting tons, and I don’t want to see that mess even if nothing is actually going bad. time_warp is working out fine.

1 Like
13

I was, yes - thinking about it again that might have been why I switched to using 0 unless I had to. Glad time_warp is working out, I might switch to that.

14

I’m thinking the devs provide time_warp as an “official” way for ordinary users to do hacky interventions into the scheduler. In that case, they’d maintain it so that there aren’t unwanted or unnatural side effects, so I see it as the safe bet. I’m just concerned that NOTE_OFF_ALLOWANCE, sustain:, or use_real_time could work out in unpredictable ways and with unexpected interactions in future versions. I mean, they’ve already mentioned this current re-working of the MIDI subsystem. It would be nice to hear the devs weigh in authoritatively on best practices and what makes them best, or maybe they haven’t reached a consensus yet.

15

Hi there,

I’m not sure what specifically you’re asking us to weigh in on here. However, I can be clear by saying that use_real_time is never a good idea unless you’re using it in a thread that’s responding to real time events - such as MIDI in. For example, if you’re writing a live loop which responds to a MIDI keyboard, you want to reduced the latency between receiving the event and producing a sound. This is where use_real_time is useful and important.

Everywhere else, use_real_time will essentially mess with the timing of your program.

I’m not sure how you see time_warp as hacky in any way. It doesn’t create any kind of interventions in the scheduler at all. All it does is modify the timestamp that is used by the scheduler - so you can make something happen slightly later than expected, or if the time is less than the default schedule ahead time (which is 0.5s) then you can use negative times so it happens slightly earlier. Of course, you can’t use time_warp times less than 0.5s as actual time travel isn’t possible. It’s only because all audio events are delayed by 0.5s that there’s a bit of time to play within for scheduling things slightly earlier.

With respect to MIDI subsystem reworking - the only thing I’m aware of is a plan to improve the MIDI port names on Linux-based systems. There is current work ongoing which will integrate Ableton Link’s clock sync technology but I can’t see how that will interfere with this behaviour at this stage.

1 Like
16

Yes, that’s what I was asking about. Thanks for clarifying the best way to conceive of the MIDI situation. Since time_warp or any timing corrections applied for the sake of achieving expected operation isn’t about music in the narrow sense, I think of those things as hacky, which means the same thing to me as computer-y (as opposed to music-y).