Minecraft from MIDI


Been using Sam’s “Sonic Minecraft” code example as part of my participation in a couple of events with kids and other curious people. Also been using MIDI controllers to drive synths on another Pi, running MODEP. The FAQ (though not that frequent, still important enough): “So, you (can) use your instrument to put blocks in Minecraft?” Hadn’t set it up this way, but it does make sense.
So, here’s my attempt at this point. It’s still not ideal, which is why suggestions would be much appreciated:

live_loop :mc_note_blocks do
  mc_message "This is Sonic Minecraft"
  with_fx :reverb do
    with_fx :echo, phase: 0.125, reps: 32 do
      x = (range 30, 90, step: 0.1).look
      y = 20
      z = -12
      mc_teleport x, y, z
      ns = (scale :e3, :minor_pentatonic)
      n = ns.shuffle.choose
      noteb=(scale 0, :chromatic)[nota]
      bs = (knit :glass, 3, :sand, 1)
      b = bs.look
      synth :beep, note: n, release: 0.1
      mc_set_block b, x+20, n-60+y, z+10
      mc_set_block b, x+20, n-60+y, z-10
      mc_set_block bb, x+noteb, noteb+y, z-3
      sleep 0.25

live_loop :sax do
  nt, vel=sync "/midi/pisound_midi_ps-233vayb/1/1/note_on"
  set :nb, nt

live_loop :drum_beats do
  sample :bd_haus, cutoff: 100
  sleep 0.5

Part of the issue is in choosing block types. Sounds like some fire blocks are just taking over, regardless of which notes are played (the idea was that the note would select the block type). Also, since the blocks appear on the side, it’s less obvious what’s going on. Not to mention that the latency is pretty high, which might have to do with my device.
Eventually, might connect the two pisound HATs through a MIDI cable (and filter the wind controller’s MIDI signal, as it’s full of active_sensing messages). Hoping the latency won’t be worse and even hoping it could be better.



This is very cool! I had a play with minecraft very early on (3 years ago!) and had some success in integrating music and block placement.

I’d like to have a play with your code, but fairly immersed in further developing my own Record / Player project at the moment.


is it possible to use minecraft pi without the raspberry pi? i saw the minecraft sonic pi tutorial in the sonic pi book, and i thought it looked pretty neat… but i’m usually just on a mac rather than on the raspberry pi. but i couldn’t figure out if you could put minecraft pi on the mac like you can put sonic pi on the mac.


AFAICT, Minecraft Pi and Mathematica for the Raspberry Pi are exclusive to these devices. Seem to remember they’re not Raspbian/Pixel exclusive (pretty sure they’re on Ubuntu MATE for RasPi). But they’re specially licensed for use with the Raspberry Pi Foundation’s device.
Maybe some of Microsoft’s versions have an API and maybe Minetest has a similar API. But this kind of “Sonic Minecraft” script was really meant for Raspberry Pi.


No worries. Been thinking about it a bit more and the best thing is probably to just focus on putting blocks from incoming MIDI and not worry about having an existing loop which could just confuse things further.
Also figured that it’s probably best to have the incoming MIDI arrive from both controllers (wind controller and keyboard) running MODEP synths. As kids will play the keyboard, they’ll surely enjoy having blocks appear in Minecraft. And, come to think of it, the “latency” is probably just due to the length of the main loop.


Ok, this works:

block_type=(ring :fence, :glass, :farmland, :door_wood, :dirt, :diamond,  :cobblestone,  :clay,  :cactus,  :chest,  :brick,  :bookshelf,  :bedrock,  :bed)

live_loop :sax do
  nt, vel=sync "/midi/pisound_midi_ps-233vayb/1/1/note_on"
  block_note=(scale 0, :chromatic)[nt]
  x = (range 20, 100, step: 0.1).look
  y = 40
  z = -20
  mc_teleport x, y, z
  mc_set_block b, x, y, z-5

It takes incoming MIDI (from another Pi, running MODEP), and puts blocks related to notes in front of the view (z-5), and moves along the x axis. Works with both a keyboard and a wind controller, plugged in at the same time (and sending on the same MIDI channel).
Will try to add some musical response on the Sonic Pi side since, after all, the idea is to showcase some musical affordances. Might bring back my countermotion script since it can be fun to play. Could also constrain notes to a scale or assign certain events to certain notes. Obviously, the possibilities are endless.
Will probably not have much time to work on this before Saturday, so having something which works is quite important for me at this point.


Ok, added a version of my countermelody script to another buffer. Latency is significant, but it makes it sound like Sonic Pi is responding to the controllers. So, quite usable.
Will probably tweak some numbers but it should be fine for Saturday.

use_real_time # Preventing latency

delta=counter=naute=idx=0 # Initialize a few things
previous=60 # First note for the counter melody
gamme=(scale 50, :hex_dorian, num_octaves: 5) # Setting a “ring” of values in which to find incoming notes
counter_gamme=(scale 50, :hex_dorian, num_octaves: 2) # Same for the countermelody, an octave below
counter_melody = synth :blade, cutoff: 100, note: 0, release: 1, amp: 2
live_loop :notes do
  note_on, velocity = sync "/midi/pisound_midi_ps-233vayb/1/1/note_on" # Incoming notes from Yamaha WX-11 wind controller
  if velocity > 0 # Only use actual note-ons
    delta=note_on - previous # Are we going up or down? By how much?
    if gamme.index(note_on) # If you find the note in the scale…
      idx=gamme.index(note_on) # Set the position in the scale
      if delta > 0 # We’re going up, so go up the scale
        idx=idx-1 # Otherwise, go down the scale
    naute=gamme[idx] # Find the note in the scale
    if delta < 0 # If we’re going down
      counter=counter_gamme.tick # Increment the counter if we’re going down
      counter=counter_gamme.reverse.tick # Decrement it otherwise
    play counter_melody, note: counter-12, amp: velocity / 160.0 # The counter note goes in reverse motion, an octave lower
    previous=naute # Set the current note for the next time around


Tried both scripts in parallel and the performance is perfectly fine. The overall effect is quite unusual as befits a Raspberry Jam. So, so far, so good.
Of course, this wasn’t a hard problem to solve. You might even say that it could be fully solved in a matter of minutes. But just like overcommenting my code, there’s value to spending the time thinking out loud about possibilities, decisions, etc.
Might also add some audio FX on the MODEP side of things. For instance, a bit of delay could blend well with the latency between a note played on a controller and the note coming from Sonic Pi (would guess it’s around 200ms or so). With a good amount of feedback on a delay line, it could all sound purposeful while remaining playful.
At any rate, will probably report back after the Jam at the museum, Saturday. (Actually, will bring my MODEP Pi to a real music jam with friends, tomorrow.)
Let’s just say Raspberry Pi is on my mind, these days.