How to diagnose midi timing problems?

BN1701 · February 14, 2021, 5:12pm

Hi,

I’ve been using SPi for a while now on Windows and Ubuntu Studio and finding it both fun and powerful. I recently decided to actually buy a Raspberry Pi as a dedicated box for running Sonic Pi to feed midi into the rest of my setup (which is just a bunch of outboard synths). I assumed that solid midi timing would be well within the capabilities of the thing, especially as I wouldn’t be using any audio or running anything else on the system. Unfortunately, this doesn’t seem to be the case. Running a simple loop sending a steady stream of midi note triggers to a drum machine, every few seconds I hear it slow briefly and then catch up. Here’s the loop:

live_loop :solo do
  use_midi_defaults port: "iconnectmidi4+_iconnectmidi4+_midi_2_24_1", channel:10
  
  midi :e0 - 12, vel: 40
  
  if (tick % 8) == 0
  midi :fs0 - 12, vel: 127
  end
  
  sleep 1.0/8
end

I’m looking for advice on how to diagnose this. I don’t think there are other processes that are hogging the CPU. Nothing else shows up when I run top, for instance, sonic-pi, ruby and beam.smp are the top three processes and between them they are using less than 50% of CPU capacity.

I installed gnome-system-monitor to show real-time graphs of not only CPU, but network and swap space, and there are no spikes or troughs associated with the bad timing events. I’ve also tried setting a higher priority for these processes and whilst I think I might be hearing more stable timing with fewer glitches, there are still glitches.

What else can I look at to get to the bottom of this? Does anyone know of other diagnostic tools that will show me, preferably in real time, any other IO activity - USB or otherwise - that might be interfering or causing a bottleneck here?

I have an RPi 4b with 8 Gb RAM and it’s running 5.10.11-v7l+ #1399 SMP Thu Jan 28 12:09:48 GMT 2021 armv7l GNU/Linux. Sonic Pi is Version 3.3.1. I’m running headless and using VNC Server/Viewer to create a desktop session for Sonic Pi. The midi interface is an iConnectMIDI4+ connected to a USB 2 port.

Any suggestions would be most welcome.

Thanks

d0lfyn · February 14, 2021, 6:59pm

Hi,

Just a random idea: does keeping time in a separate live_loop make a difference?

live_loop :solo do
  use_midi_defaults port: "iconnectmidi4+_iconnectmidi4+_midi_2_24_1", channel:10
  
  sync :play;

  midi :e0 - 12, vel: 40
  
  if (tick % 8) == 0
    midi :fs0 - 12, vel: 127
  end
end

live_loop :keepTime do
  cue :play;
  sleep 1.0/8;
end

Sorry if this isn’t helpful. I hope you figure it out!

robin.newman · February 14, 2021, 7:33pm

I’ve tried your program using an identical Pi4 with 8Gb ram running SP 3.3.1 and my external usb midi-interface (a Steinberg UR22 Mkii). I adjusted the notes to fit the drumkiot on my Korg synth. Seems to work fine with no deviation from the expected rhythm over 15 minutes. I also ran it via a VNC connection in case that was causing the problem.
The Pi4 has two USB2 and 2 USB3 intrfaces. Have you tried both? (the usb3 are blue)

BN1701 · February 14, 2021, 8:03pm

A good thought, because I actually started off with the master timing thread idea - that’s how I usually run things - but I simplified it to the fragment I posted here. The timing glitches occurred with both situations. Thanks, though

BN1701 · February 14, 2021, 8:06pm

Thanks Robin, that’s encouraging. (And disappointing if it turns out to be the interface.) I have some other midi interfaces I can try. I’ll report back in a while. (I’ll try the USB 3 port, too, though the iConnectivity box is only a USB 2 device.)

samaaron · February 14, 2021, 8:19pm

One approach is to get the MIDI stream into a DAW and record the events (as if you were recording MIDI events from a connected piano) . Then you can examine the MIDI score to see the exact timings.

Another is to loopback the MIDI back into Sonic Pi and print out the times of the incoming cue events.

soxsa · February 15, 2021, 7:49am

Hi @BN1701 I’m using the same setup idea, aside from the specific hardware. I have a RPi4 4 or 2Gb and MidiTech MIDIface 4x4 USB hub I’ve found mine to be highly stable, both sending midi notes and midi clock. I mean rock solid.

Sounds like you’re doing the right things, so I’ve not got an answer I’m afraid. When you ran SPi from another PC (Windows or Mac) I assume you got a stable result, is that right?

The only thing I’d throw in is the ‘8Gb’ Rpi bit. I don’t have that, but I’ve read elsewhere various reports of things (other things than Spi I mean) not working right on the 8Gb model - no idea why that is. Spi doesn’t need anything like that btw, I originally bought 4Gb but it runs very well on the 2Gb model.

Very interested to hear what happens with this, as I’m rather relying on this setup!

Just to add, if it were me, I’d look at the power consumption of your midi hub. It looks like a quality item, but wondering if the RPi can’t supply enough power. Seems unlikely but it has to be something.

BN1701 · February 15, 2021, 1:23pm

The iConnectMIDI4+ has its own power supply. I’ve had it years and it’s been solid up till now. I didn’t rule it out immediately as a culprit, though, as it’s just had a firmware update. However, I’ve now tried this with a competely different midi interface and there are still issues. (I’ll post more on this in a while.)

Thanks

B

BN1701 · February 15, 2021, 5:41pm

Thanks for all the suggestions but so far I still haven’t got to the bottom of this. (I like a puzzle, so I’m not giving up yet.)

@robin.newman, I’ve now tried another interface. Specifically, a Midisport 8x8. It’s old, I know, and I had to install midisport-firmware for it to run, but I get exactly the same behaviour as with the iConnectivity box. In fact, I’ve tried the Midisport as the only device connected to the Pi, and connected to another USB port in parallel with the iConnectivity box with the same midi data sent to synths connected to each. Both synths showed exactly the same timing wobbles at exactly the same times. I’ve also tried them both in the USB3 ports, as you also suggested.

Following up on @samaaron’s suggestion. I sent a third midi stream to a DAW (Reaper) running on a Windows PC. (This was via another port on the iConnectivity box, so I admit that this connection is not totally independent of the other two.) I was also able to record the audio from each synth so that I can check what was happening in the midi stream when I hear a timing wobble. The timing glitches seem to affect note-ons and note-offs independently. I had thought that maybe messages were being dropped, but in the couple of examples I looked at closely the note-off arrives immediately after the note-on, so the notes are just made to be very short. In the two examples below, the first shows a very late note-on followed by a note-off only slightly late. The stream stabilizes pretty quickly after that. The second shows a much more prolonged wobble lasting several ticks. (These are eighth-notes at the default tempo of 60 bpm, so the nominal time difference between each is 0.125 seconds.)

N-on	(Diff)	N-off	(Diff)
79.655	-       79.705	-
79.780	0.125	79.830	0.125
79.905	0.125	79.955	0.125
80.030	0.125	80.080	0.125
80.155	0.125	80.205	0.125
80.280	0.125	80.330	0.125
80.405	0.125	80.455	0.125
80.575	0.170	80.585	0.130
80.655	0.080	80.705	0.120
80.780	0.125	80.835	0.130
80.905	0.125	80.960	0.125
81.030	0.125	81.085	0.125
			
N-on	(Diff)	N-off	(Diff)
4.520	-       4.570   -	
4.645	0.125	4.695	0.125
4.770	0.125	4.820	0.125
4.985	0.215	4.985	0.165
5.020	0.035	5.075	0.090
5.145	0.125	5.195	0.120
5.270	0.125	5.325	0.130
5.395	0.125	5.450	0.125
5.520	0.125	5.570	0.120
5.645	0.125	5.695	0.125
5.775	0.130	5.825	0.130
5.895	0.120	5.945	0.120
6.020	0.125	6.075	0.130
6.145	0.125	6.195	0.120
6.270	0.125	6.320	0.125

The interval between wobbles is not consistent. Usually the interval is between five and twenty seconds, but I’ve had one stream of up to a minute. By the way, I see absolutely no variation in the timings reported by SPi in the log window as midi messages are sent.

Given that these glitches consistently appear on multiple outboard devices, I still have the feeling that something in the OS is interfering with hardware IO and I’d still like to be able to monitor what’s happening at that level, but I’m still looking to find out how.

Thanks

BN

soxsa · February 15, 2021, 6:50pm

Excellent diagnostics. Obvious questions like is the Rpi patched to the latest version, have you installed anything else unusual? Sounds like you don’t need us to suggest these kind of things, just throwing some ideas around. Again, if it were me I’d flash another OS from scratch and start vanilla.

Could be a defective USB controller or Rpi. How about the 8Gb vs 4/2Gb problems I’ve read about. Vague I know but worth a look.

Just saying that I had a wierd jack/audio problem in Rpi last week, and after a long trip around the block the problem was I was connecting via RDP instead of VNC. Who knew? (I know you are using VNC so I’m not suggesting it’s that, just that it can be something left-field).

Time vs money, for £34 I’d personally try another Rpi - but not everyone’s cup of tea I know. For all the little difficulties I’ve had with audio on the Rpi, MIDI has been constantly good throughout.

I feel your pain!

Oh, how about downgrading Spi to 3.2? 3.3 has only just come out.

Run a DAW on the Rpi sending out midi, see if that’s ok? Is there one?

Plug a usb midi keyboard into the RPi, and use QJackCtl to route the midi out to your midi hub - see if that gives timing errors. Looking to see if it’s Spi or Rpi to blame.

BN1701 · February 15, 2021, 6:59pm

Two things you said immediately spark further thoughts … finding something else to generate a stream of midi pulses. It may not even be S-Pi at all.

And audio … It occurred to me to have S-Pi actually play notes using internal synths instead of sending midi. As I’m running headless I haven’t even bothered to try and get audio working up till now. Except, sound isn’t working at all on the thing at the moment. Not just S-Pi, but Pulse Audio reports no devices devices and Qjackctl reports startup errors which I’ve still to investigate. I feel another trip round the block coming on …

soxsa · February 15, 2021, 7:07pm

With the audio, there’s info on here to get it working with pulseaudio and the standard AV headphone socket. Myself I’m using jack/alsa with a USB-connected audio box - which gives much better latency. This is particularly because I’m using SPi to create sounds AND send midi hither and thither. And there’s info on here about that too.

I’m feeling my way with Linux Audio, sounds like you’ve got more experience so I’ll defer to you and the knowlegeable people on here! Just to say that they pointed me in the right direction with this.

In jack, there are options for selecting midi systems too - I’d love to understand it but have gone with what works, and it works well. I’m just thinking that might be related to your midi woes too.

soxsa · February 15, 2021, 7:18pm

If it’s any use, this is what my QJackCtl Connections looks like when SPI is running

samaaron · February 15, 2021, 9:25pm

Another thing to try is to increase the priority of both the Ruby (ruby) and Erlang (beam.smp) processes using renice. For example:

sudo renice -18 298374

(where 298374 is the process id of either Ruby or Erlang).

robin.newman · February 17, 2021, 9:06pm

I’ve been having a play with feeding the midi back into Sonic Pi and using it to play notes. I think that there may be a potential problem here.
One thing that occurs to me is that you are not explicitly defining the length of the notes you are sending. If you do not then each note (time between midi on and midi off is 1 beat, and since you don’t explicltly set the tempo that will be 1 second. So you will have lots of notes stacking up.
I think you are driving percussive outputs which probably only need the note_on to trigger. have you tried setting a very small sustain value, less than the loop time which is 1.0/8? Does that make any difference?

I have now used the program below on both RPi and mac. Both have jitter in third decimal place. I think it improves if you reduce the sustain time of the midi note sent. I haven’t played the received note, just worked out the timing separations.

use_debug false
use_midi_logging false
use_cue_logging false
set :kill,false
#change the following port to suit your system
#here set for mac virtual port
#on RPi use midi through port
use_midi_defaults port: "iac_driver_sonic-pi",channel: 1
use_bpm 60
live_loop :msend do
  use_real_time
  midi :c4 #,sustain: 0.0625
  sleep 0.125
  stop if get(:kill)
end
t=0 #set initial time offset
live_loop :min do
  use_real_time
  n = sync "/midi*/note_on"
  puts (vt-t).round(3) #print delta time EDIT ADDED ROUNDING
  t=vt #missed out. put back in edit
end

at 10 do #run for seconds then stop
  set :kill,true
end

Look at the times in the log when run.
Compare with the sustain time uncommented.

soxsa · February 15, 2021, 10:27pm

Sounds like you might have it. Would it be an idea try that with Spi running on the original Windows and Ubuntu boxes and see if it behaves the same?

BN1701 · February 15, 2021, 11:01pm

@robin.newman I actually have been setting sustain, though it doesn’t show in the original code snippet that I posted. I’m not sure sure why it got left out, I probably messed up when I was working out how to post a code fragment. Sorry about that if it is sent you off chasing red herrings. For confirmation, here’s the exact loop that I used to generate the timing data I posted a few messages back:


offset = 12


live_loop :solo do
  use_midi_defaults sustain: 1.0/16
  
  
  ##| Two synths ...
  midi :e4 + offset, vel: 20, port: "midisport_8x8_midisport_8x8_midi_1_24_0", channel:11
  midi :e4 + offset, vel: 20, port: "iconnectmidi4+_iconnectmidi4+_midi_3_20_2", channel:2
  
  ##| Log to DAW (Reaper on Windows) ...
  midi :e4 + offset, vel: 20, port: "iconnectmidi4+_iconnectmidi4+_midi_16_20_15", channel:1
  
  if (tick % 8) == 0
    # stop
    midi :fs4 + offset, vel: 127, port: "midisport_8x8_midisport_8x8_midi_1_24_0", channel:11
    midi :fs4 + offset, vel: 127, port: "iconnectmidi4+_iconnectmidi4+_midi_3_20_2", channel:2
  end
  
  sleep 1.0/8
  
end

Meanwhile, I’ll have a go with your code and see what there is to see.

BN1701 · February 17, 2021, 4:36pm

Hi @robin.newman, a day later than hoped, but I’ve had a look at your code. Firstly, it looks like there’s a line missing in the :min loop just before end. I’m assuming there should be t=vt at that point. If so, results are interesting. Here’s a section of output from running on my Pi:

(Sorry it’s a screen grab, this version of SPi seems to have lost the Right-Click menu over the log window. At least, it’s there when SPi starts up but seems to become disabled after the log window is written too. This is the same on Windows and the Pi. Either that’s a bug, or I’m being stupid.

Edit: I was being stupid. It seems you can’t right click on the log window whilst there are active threads. Making sure all threads have stopped re-enables selection and the Right Click Menu.)

Anyway, you can see that timing is mostly solid but there’s a big wobble around the fourth and fifth entries. This is with the short sustain setting reinstated. Also interesting is that when I run this code on my Windows PC overall timing is actually more variable, though there are no big wobbles like the ones I see here.

@samaaron I also tried your renice suggestion for both ruby and beam.smp, but I didn’t notice any difference in behaviour.

Overall, I’m still dubious about the (lack of) setup of audio on my Pi and I still haven’t had time to get that working. One thought is that I’m using VNC virtual sessions and of course, there’s no desktop audio support there at all. Jack startup fails, too. I don’t know, however, what effect this might have on SPi. (I switched to virtual VNC sessions because of the flexibility of desktop sizes. When I started off with sharing the physical desktop it seemed stuck on something small and none of the recommended procedures for changing it seemed to work. Must try harder … )

BN

robin.newman · February 17, 2021, 9:07pm

Oops about missing line. I’ve edited the code to put it back again. I tried with and without vnc. Didn’t make discernable difference for me.

BN1701 · February 19, 2021, 5:17pm

Update …

I have audio working on my Pi, still headless. (Long story, but when you convince VNC that there’s a real screen attached with either hdmi_force_hotplug or hdmi_ignore_edid in config.txt you can plug in some headphones.) Good news, actually hearing a string of notes using play sounds rock solid. Not so good news, this makes no difference to the midi timing wobbles. (I’d hoped that somehow deep down in the gubbins of SPi having a stable audio stream might somehow give the midi stream something to lock onto.)

Following up on @soxsa’s suggestions, I tried another app sending midi. In fact I created a test stream in a DAW (Reaper), exported it to .mid and played it out with aplaymidi. Timing was again rock solid.

I also went back to SPi V3.3 on the Pi. The timing wobbles are present here, too. I tried to go back further, but SPi 3.2.2 gave a server boot error on this Pi.

For extra info, I double checked with @robin.newman’s test code on Windows. With just a visual check I see timing variations of +/- 0.006 seconds for most notes.This seems alarming, but when I went back and checked with SPi 3.2.2 I found that numbers are comparable and since I’ve been happily using that version since I first discovered SPi I guess it’s not a problem for me. On the Pi, however, ignoring the big wobbles, this value is +/- 0.001, and I realy can hear the difference. (This is also comparable to when I run SPi 3.3 beta on Ubuntu Studio, though I don’t have 3.3.1 working there yet).

Anyway, all this still really makes me want to get to the bottom of things …

BN