Inconsistent SysEx behaviour between Windows and Linux

I’m completely stumped by this problem and I would really appreciate your advice.

I’ve written a Sonic Pi programme that sends SysEx messages to display patterns on an array of LEDs (a Novation Launchpad Mini Mk2 connected via USB). I’ve been developing and using it for a few weeks, without any issues, on Sonic Pi v4.4.0 on 64-bit Windows 10.

I have just moved the code onto a Raspberry Pi 4 Model B - running Sonic Pi v4.3.0 on 32-bit Raspbian 11 (bullseye) - and it doesn’t work at all. Other aspects of the code - live audio and MIDI controlled synths - are running perfectly. But the frames of SysEx animation do not display on the LEDs and, halfway through sending a stream of them, eventually display a single, jumbled frame of the data.

I have no idea where to start fixing this - it works perfectly on my Windows machine. I’m a Linux noob and I’m hoping there is some setting or fix that I’m not aware of.

Update:

I just tried a completely fresh install of 64-bit Raspbian 11, on a different SD card, so I could try running my code in Sonic Pi v4.3.0 - the same version as my Windows system.

The problem is the same: on the Raspberry Pi, the SysEx messages don’t work. I’ve checked the log, and the code is generating the exact same message - it just doesn’t seem to be getting out onto the USB MIDI device properly.

Are you specifying the midi port to use? Otherwise it transmits on all ports which may be a problem. Does the port you want to use show up in the io preferences?

Thanks Robin, yes I’m specifying a port for the messages using the same name as is displayed in the IO preferences. I was wondering if the ‘midi through’ that appears there on Raspbian might be a problem, but I’m using the port number and not sending to/from that anyway.

This might be way too much info, but here’s a condensed example of the code - it lights up LEDs on an 8x8 grid. The draw function takes a number of arrays as arguments - each containing a colour code and a list of LEDs to light - and it uses these to fill in a default array which is otherwise populated with blanks (represented by the number 12 here).

All of these values are then used to make and send a special SysEx message in the following format. This is taken from the programmer guide to the Launchpad Mini mk2 which describes the SysEx it expects as follows:

Rapid LED update

Hex version:
92h, Velocity 1, Velocity 2,
92h, Velocity 3, Velocity 4 …

Decimal version:
146, Velocity 1, Velocity 2,
146, Velocity 3, Velocity 4 …

Sending a MIDI channel 3 note-on message enters a special LED update mode. All eighty LEDs
may be set using only forty consecutive instructions, without having to send any key addresses.
Irrespective of the mapping chosen, this will update the 8x8 grid in left-to-right, top-to-bottom
order, then the eight scene launch buttons in top-to-bottom order, and finally the eight
Automap/Live buttons in left-to-right order (these are otherwise inaccessible using note-on
messages). Overflowing data will be ignored.

To leave the mode, send a standard messsage beginning with 80h, 90h, or B0h. Sending another
kind of message and then re-sending 92h will reset the cursor to the top left of the grid.

So my Sonic Pi draw function looks like this:

use_real_time

# Specify the device as a port for sending sysex
# (optional, but otherwise sysex sends to all midi ports)
set :lp, "2-_launchpad_mini_2"
set :rest_time, 0.25

# Define the frame-drawing function
define :draw do |*args|
  
  # establish defaults - each LED defaults to blank
  norm=[12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,]
  
  # Sort through the input - each array given starts with a number for the colour
  # and is followed by a list of LEDs to set to that colour 
  args.each do |stuff|
    # take each number and map it
    stuff.drop(1).each do |led|
      # add it to the right variable for the upcoming sysex
      norm[led] = stuff[0]
    end
  end
  
  # send a single sysex with the image
  midi_sysex 0xF0,146,12,12,
    146,norm[0],norm[1],146,norm[2],norm[3],146,norm[4],norm[5],146,norm[6],norm[7],
    146,norm[8],norm[9],146,norm[10],norm[11],146,norm[12],norm[13],146,norm[14],norm[15],
    146,norm[16],norm[17],146,norm[18],norm[19],146,norm[20],norm[21],146,norm[22],norm[23],
    146,norm[24],norm[25],146,norm[26],norm[27],146,norm[28],norm[29],146,norm[30],norm[31],
    146,norm[32],norm[33],146,norm[34],norm[35],146,norm[36],norm[37],146,norm[38],norm[39],
    146,norm[40],norm[41],146,norm[42],norm[43],146,norm[44],norm[45],146,norm[46],norm[47],
    146,norm[48],norm[49],146,norm[50],norm[51],146,norm[52],norm[53],146,norm[54],norm[55],
    146,norm[56],norm[57],146,norm[58],norm[59],146,norm[60],norm[61],146,norm[62],norm[63],
    146,norm[64],norm[65],146,norm[66],norm[67],146,norm[68],norm[69],146,norm[70],norm[71],
    146,norm[72],norm[73],146,norm[74],norm[75],146,norm[76],norm[77],146,norm[78],norm[79],
    0x80,0x00,0xF7, port: get[:lp]
  
  # wait a given period of time before the next frame of animation (usually 0.5 or so)
  sleep get[:rest_time]
  
end

And it is called with little functions like this - a simple number of consecutive frames:

# Simple animation with a face
define :face do
  draw [13, 17, 22], [28, 51, 52, 50, 53, 43, 44, 45, 42]
  sleep 2
  draw [13, 8, 13], [28, 44, 43, 42, 45, 52, 51, 53, 50]
  sleep 1
  draw [13, 10, 15], [28, 44, 43, 42, 52, 50, 51, 53, 45]
  sleep 1
end

This works perfectly on my Windows system - with some very long, detailed, and fast animations. They update to the LEDs smoothly, without any problems or inconsistencies.

When I try the same code on the Raspberry Pi, the messages appear to be sending as normal but the display doesn’t light up at all. If I run it once or twice in a row, eventually a garbled version of one of the frames will appear - with the image shifted to the side like the coordinates are incorrect. It makes me think that the SysEx messages are either not getting through, only partially arriving, or being sent in some erratic way. I have also tried slowing things down dramatically, leaving seconds between each frame, and the results are the same.

This is very intertesting. I have had a play with it this morning, and I think I have got it working.

It appears that the midi behaviour on Raspberry Pi (and indeed on a linux based versions) doesn’t behave quite properly. It is implemented in a different way, and this probably gives rise to the problem.

I tried your code on both Raspberry Pi4 and a vritual Debian 12 running on my Mac and connected up a midi monitor snoopmidi (which you can install using sudo apt install) to monitor the output. (You have to configure it not to ignore sysex events). As you describe I got intermittent results, even though the messages were displayed in the SP log. I played around with it and eventually found that if I sent a standard midi message just before the sysex call it worked. I tried this inserting a midi_note_off 0 just before the midi_sysex in your draw function, and it then seemed to work OK. I also tried putting in an arbitrary midi_cc call instead of the midi_note_off and that worked too.

I tried this on the Mac version, which worked OK to start with, and it still did with the extra bodge call added.

I will be interested to see if this works OK with the real haredware.
Note I also specified the port and a channel in a use_midi_defaults line, to reduce the extranoujs calls that would happen if the channel wasn’t specified.

Amended draw function

# Define the frame-drawing function
define :draw do |*args|
  
  # establish defaults - each LED defaults to blank
  norm=[12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,]
  
  # Sort through the input - each array given starts with a number for the colour
  # and is followed by a list of LEDs to set to that colour
  args.each do |stuff|
    # take each number and map it
    stuff.drop(1).each do |led|
      # add it to the right variable for the upcoming sysex
      norm[led] = stuff[0]
    end
  end
  midi_note_off 0 #bodge to get linux SP working
  # send a single sysex with the image
  midi_sysex 0xF0,146,12,12,
    146,norm[0],norm[1],146,norm[2],norm[3],146,norm[4],norm[5],146,norm[6],norm[7],
    146,norm[8],norm[9],146,norm[10],norm[11],146,norm[12],norm[13],146,norm[14],norm[15],
    146,norm[16],norm[17],146,norm[18],norm[19],146,norm[20],norm[21],146,norm[22],norm[23],
    146,norm[24],norm[25],146,norm[26],norm[27],146,norm[28],norm[29],146,norm[30],norm[31],
    146,norm[32],norm[33],146,norm[34],norm[35],146,norm[36],norm[37],146,norm[38],norm[39],
    146,norm[40],norm[41],146,norm[42],norm[43],146,norm[44],norm[45],146,norm[46],norm[47],
    146,norm[48],norm[49],146,norm[50],norm[51],146,norm[52],norm[53],146,norm[54],norm[55],
    146,norm[56],norm[57],146,norm[58],norm[59],146,norm[60],norm[61],146,norm[62],norm[63],
    146,norm[64],norm[65],146,norm[66],norm[67],146,norm[68],norm[69],146,norm[70],norm[71],
    146,norm[72],norm[73],146,norm[74],norm[75],146,norm[76],norm[77],146,norm[78],norm[79],
    0x80,0x00,0xF7, port: get[:lp],channel: 1
  
  # wait a given period of time before the next frame of animation (usually 0.5 or so)
  sleep get[:rest_time]
  
end
1 Like

Thanks so much for looking into this Robin - I was literally about to try to redesign the whole thing without SysEx when you posted just now! I’m going to try this and I’ll let you know :slight_smile:

Wow, this is it Robin! I installed midisnoop and did a load of tests observing the messages and different placements of the bodge midi_note_off 0. It seems that the SysEx messages get stuck in transit until the note_off hits it and releases them. For me, it works when I put the note_off after the SysEx call. Putting it beforehand seems to work, but only (I’m guessing) because consecutive attempts to run the code allow the note_off to release the SysEx from the previous run.

(Also, I had to change the note_off to an arbitrary value of 9 - because certain notes can be used to turn on/off the LEDs individually on the Launchpad and 9 is not one of them)

I’ve observed a few more quirks…using a note_off after the SysEx does not work for me unless I also implement a sleep command. It doesn’t seem to matter how long, but a brief (0.001) sleep between the SysEx and the note_off seems to do the trick. So I was able to get consistent results with code like this:

draw [13, 0, 1, 2] #lights up the first 3 LEDs with colour 13 / red
sleep 0.001
midi_note_off 9, channel:1, port: get[:lp]

draw [60, 3, 4, 5] #lights up the next 3 LEDs with colour 60 / green
sleep 0.001
midi_note_off 9, channel:1, port: get[:lp]

This seemed to display everything at pace the images were all confused - like the coordinates were shifted to the right. So I made the following change in the section that maps each number from the argument arrays to the norm[] array for the SysEx:

# add it to the right variable for the upcoming sysex - NB this has been changed
      norm[led-2] = stuff[0]

This fixed things, and the images displayed perfectly! However, it got me wondering why…and I looked at the SysEx formatting and noticed the first line had an extra 146,12,12, that I had introduced back when I was getting it working on Windows. On Linux, this seemed to be giving 2 blank LEDs at the start, which I was then rolling back with the norm[led-2] above, so I removed both and voila! Everything is running as it should!

Finally, I added the tiny sleep and the midi_note_off into the draw function itself. Here’s the final code, which works with all the animations I’ve been making at last:

# Define the frame-drawing function
define :draw do |*args|
  
  # establish defaults - each LED defaults to blank
  norm=[12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,
        12,12,12,12,12,12,12,12,]
  
  # Sort through the input - each array given starts with a number for the colour
  # and is followed by a list of LEDs to set to that colour
  args.each do |stuff|
    # take each number and map it
    stuff.drop(1).each do |led|
      # add it to the right variable for the upcoming sysex 
      norm[led] = stuff[0]
    end
  end

  # send a single sysex with the image
  midi_sysex 0xF0,          # extra 146, 12, 12 removed
    146,norm[0],norm[1],146,norm[2],norm[3],146,norm[4],norm[5],146,norm[6],norm[7],
    146,norm[8],norm[9],146,norm[10],norm[11],146,norm[12],norm[13],146,norm[14],norm[15],
    146,norm[16],norm[17],146,norm[18],norm[19],146,norm[20],norm[21],146,norm[22],norm[23],
    146,norm[24],norm[25],146,norm[26],norm[27],146,norm[28],norm[29],146,norm[30],norm[31],
    146,norm[32],norm[33],146,norm[34],norm[35],146,norm[36],norm[37],146,norm[38],norm[39],
    146,norm[40],norm[41],146,norm[42],norm[43],146,norm[44],norm[45],146,norm[46],norm[47],
    146,norm[48],norm[49],146,norm[50],norm[51],146,norm[52],norm[53],146,norm[54],norm[55],
    146,norm[56],norm[57],146,norm[58],norm[59],146,norm[60],norm[61],146,norm[62],norm[63],
    146,norm[64],norm[65],146,norm[66],norm[67],146,norm[68],norm[69],146,norm[70],norm[71],
    146,norm[72],norm[73],146,norm[74],norm[75],146,norm[76],norm[77],146,norm[78],norm[79],
    0x80,0x00,0xF7, port: get[:lp],channel: 1

    # Tiny gap between the sysex and the bodge
    sleep 0.001

    # Note off to 'push' the sysex above through 
    # (for some reason, only channel 1 works - maybe a quirk of the hardware)
    midi_note_off 9, channel:1, port: get[:lp]
  
  # wait a given period of time before the next frame of animation (usually 0.5 or so)
  sleep get[:rest_time]
  
end

I haven’t tested it back on Windows yet, to see if the coordinates will be weird, but I’ll get around to it.

I really appreciate your help with this - I would have never figured out what to do without zooming in on the midi getting stuck that you discovered. I know that this is all pretty abstract right now, but I’ll be happy to share some footage of the final piece working once it’s all sorted out!

Hopefully this will all continue to work predictably as I finish this piece :crossed_fingers:

EDIT: Strangely, the only difference now running this on Windows is that the LED array seems 2 entries behind. A simple fix, now that I know what I’m looking for, just need to add +2 on Windows but not on Raspbian

That’s great. I’ll see if I can spot anything in the source code, but I think it’s one that @samaaron will have to sort out as it may be in the nitty gritty of the tau code. There is certainly something odd going on.

1 Like