Sonic Pi visualiser written in P5.js

Creations & Ideas
1

I have spent the last few days developing a visualiser for Sonic Pi written in P5.js This is my first foray into P5.js although I have used processing before. Quite a steep learning curve to get all the bits workig together, especially working out how to add OSC support. I’m quite pleased with the end result which has around 40 parameters controllable via OSC messages. I have developed a TouchOSC template which can be used to control it, but you can also include OSC commands sent from Sonic Pi.

IMG_1589
IMG_15892048×1536 178 KB

Here is a piece which uses a folder of percussion samples which I wrote some time ago, but have now added OSC commands to it to control the visualiser.

It would be a large task to do a write up on this, as there is a lot invloved, but let me know if you are interested and I’ll see if I can do something.

here is a listing of the Sonic PI end for this piece.


#piece to test out my Sonic Pi visualiser, controlled from Sonic Pi.
use_osc "192.168.1.128",12000
osc "/1/transShape",1
osc "/1/transStar",0.35
osc "/1/starVol",0.6
osc "/1/jitterVol",1
use_random_seed 987123
kill=0
with_fx :level do |v|
  
  control v,amp: 0
  sleep 0.02
  in_thread do
    loop do
      tick
      control v,amp: look.to_f/40 if look <40
      control v,amp: 1-[(look-400).to_f/40,1].min if look >400
      sleep 0.2
      kill=1 if look >440
      stop if look >440
    end
  end
  
  zDrumSamplesPath="/Users/rbn/Downloads/DrumsetSamplesFlac"
  with_fx :gverb,room: 25 do
    live_loop :drmrave do
      tick
      use_bpm 30+rand_i(90) if look%20==0
      sample zDrumSamplesPath,rrand_i(35,81)-35, rpitch: rrand_i(-24,24),amp: rrand(0.4,1.8),pan: rrand(-1,1)
      sleep rrand(0.05,0.2)
      stop if kill==1
    end
  end
end

live_loop :oscControl,sync: :drmrave do
  tick
  tick_set :t4,look/4
  c=rand_i(8)
  osc "/1/pt"+c.to_s,1 if c > 1
  osc"/1/allOff",1 if c==1
  j=(ring 0,1).look(:t4)
  osc "/1/jitter",j
  osc "/1/colInvert",rand_i();
  ilist=['/1/inc0','/1/inc2','/1/inc4']
  osc ilist.choose,1
  alist=['/1/angle45','/1/angleNeg45','/1/angle30','/1/angleNeg30','/1/resetAngle']
  if j==0
    osc alist.choose,1
  else
    osc "/1/resetAngle",1
  end
  
  osc "/1/smallStarEnable",rand_i();
  sleep 1
end
7 Likes
2

This is great! I’ve been trying to get p5 and Sonic Pi to communicate for a while.
How were you able to send OSC messages into P5? Was there a library you had to add?
Can you please share the p5 code you used for this as well?

Thanks

3

Yes I found a library that worked, but it is quite involved in getting it going. It requires a bridging node script to be run before each connection that you make.
I will be publishing the code when I’ve finished fiddling with it. I also want to see if I can get it going on a Raspberry Pi as well.
The OSC library I used is here

1 Like
4

Thanks. I had come across the same library but didn’t really have the know-how to get it to work. Hopefully getting to see your code will offer some insight.

5

@robin.newman this stuff is really amazing. I would love to see the code for that dashboard/panel you have for the whole visualization system. Do you have any of the p5.js code on Github?

I’ve been trying to do some basic browser visualizations with vanilla JS based on some OSC.js examples I found over the weekend.

This is a kind of matrix-sequencer-looking grid visualizer I made for one of my youtube videos:

(16x8 matrix visualizer code here)

The code isn’t too gnarly (without p5.js) using the DOM api, CSS and event listeners (example here)

@mrbombmusic, if you are looking to connect SonicPi this is a pretty minimal boilerplate I made that you are welcome to use or iterate on:

// data container
let data = [];


// OSC.js stuff
const handleMessage = (msg) => {
    // console.log('MSG', msg); // debug the raw OSC message
    data = msg.address.split('/');  // split string on slashes
}

const initOSC = () => {
    // Init container

    // Init port
    oscPort = new osc.WebSocketPort({
        url: "ws://localhost:8081"
    });


    // listen
    oscPort.on('message', (msg) => {
        handleMessage(msg); // Debugging
    });
    
    // open port
    oscPort.open();
};

// used later to start OSC
window.initOSC = initOSC();

// P5 stuff -- or whatever library, like WebGL or just raw Canvas calls
let colSize = 0;
let rowSize = 0;
let lastDraw = 0;
const MAX_DURATION_OF_BLINK_MS = 100;
const MAX_COLS = 16;
const MAX_ROWS = 8; 
const COLORS = [
    [255,204,0],
    [255,0,0],
    [0,255,0],
    [0,0,255]
]

function setup() { 
    createCanvas(windowWidth, windowHeight);
    colSize = windowWidth/MAX_COLS;
    rowSize = windowHeight/MAX_ROWS;
} 

function draw() { 
    // When there is new data
    if (data.length) {
        // Extract data from global variable
        const [,,a,b,c] = data;
        const numA = Number(a); // col
        const numB = Number(b); // row
        const numC = Number(c); // color

        // Draw colored rect
        fill(...COLORS[numC]);
        rect(numA * colSize, numB * rowSize, colSize, rowSize);

        // This is basically a timer
        const now = millis(); // check current time
        // compare to last drawn frame
        if (now - lastDraw > MAX_DURATION_OF_BLINK_MS) {
            lastDraw = now;
            data = [];
        } 
    } else {
        clear();
    }
}

Here’s a generic SonicPi script that drives the above JS code:

T = 4.0

use_osc "0.0.0.0", 57121 # you can get this IP from the output of the node server

kick_patterns = [
  (bools, 1,0,1,0, 0,0,1,0, 0,0,1,0, 0,0,1,0), # Kick Pattern 1
  (bools, 1,0,0,0, 0,0,1,0, 0,1,1,0, 0,1,1,0) # Kick Pattern 2
].ring

snare_patterns = [
  (bools, 0,0,0,0, 1,0,0,0, 0,0,0,0, 1,0,0,0), # Snare Pattern 1
  (bools, 0,0,0,0, 1,1,0,0, 0,0,0,0, 1,0,1,0)  # Snare Pattern 2
].ring

live_loop :kicks do
  p = kick_patterns.choose
  16.times do |n|
    sample :bd_mehackit if p.tick
    # This osc event becomes the "data" array in the JS code by splitting on slash chars
    osc "/blink/#{n}/#{n/2}/2" if p.look
    sleep T/16
  end
end

live_loop :snares do
  p = snare_patterns.choose
  16.times do |n|
    sample :sn_dolf if p.tick
    osc "/blink/#{n}/#{n/2}/3" if p.look
    sleep T/16
  end
end

<!DOCTYPE html>
<html>
    <head>
        <title>osc.js + p5.js demo</title>
        <meta charset="UTF-8" />
        <script src="/node_modules/osc/dist/osc-browser.js"></script>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.6.1/p5.min.js"></script>
        <script src="/p5js-demo/p5demo.js"></script>
    </head>

    <body>
        <script>
            window.initOSC();
        </script>
    </body>
</html>

You would also need a minimal node server to handle some of the syncing between SonicPi.

It’s still pretty rough and not quite in library form, but if it helps, I’ve packaged together a lot of this code and put it up on github:

3 Likes
6

One of my first thoughts when I came across the concept of live coded music (just a few months back now) was the potential for closely integrated dynamic visuals. So cool to see this coming to life!

7

It has been great to see all the ideas generated here, although I must admit I have not yet had time to study them closely as I have been doing further work on developing my code. I have now published a github repository containing two examples, and hope to add my larger example with TouchOSC support as well as interface to Sonic Pi shortly. (hopefully later today) EDIT THIS HAS NOW BEEN ADDED
In the meantime the reposiitory is here

I am very new to using P5.js and using it with OSC, but hope others may find the code has useful ideas.
One of the main problems is setting up the audio paths, and so far I have only just done this on a Mac, but I am currently also looking at doing this on a Raspberry Pi with a pi-sound board fitted.

2 Likes
8

As a complete noob, where would I begin in order to learn how to write my own visuals? I was originally looking (very briefly) at getting my head around shaders and the GLSL and ISF formats. However, it looks like P5.js is a totally different (possibly simpler) approach?

1 Like
9

Hi Aiden
This was my first foray into P5.js Things I found useful to get me going were:

The p5.js website, which has lots of examples and good reference sections. Look especially at the sound library examples.

Some videos by Daniel Shiffman on youtube, althouhg I find is jokey style very irriatating. However the content is useful.
eg https://www.youtube.com/watch?v=jEwAMgcCgOA and other videos in that series.

Some nice examples at https://github.com/therewasaguy/p5-music-viz

I think you want a good interactive editor that will let you quickly launch and try out sketches. I used the free Brackets Editor from brackets.io To tame it I had to modify the preference in brackets.json to

{
    "brackets-eslint.gutterMarks": true,
    "brackets-eslint.useLocalESLint": false,
    "fonts.fontSize": "14px",
    "fonts.fontFamily": "'SourceCodePro-Medium', ＭＳ ゴシック, 'MS Gothic', monospace",
    "language": {
        "javascript": {
            "linting.prefer": [
                "JSHint"
            ],
            "linting.usePreferredOnly": true
        }
    },
    "jslint.options": {
        "devel": true
    },
    "linting.JSLint.collapsed": false,
    "debug.showErrorsInStatusBar": true
}

This was to suppress some the error warnings the VERY strict jslint flags up.

Other than that googling is very useful. eg searching for
resizing p5.js sketches
returned the useful link https://github.com/processing/p5.js/wiki/Positioning-your-canvas

To get the OSC support to connect to Sonic Pi I used this library:

It was a little tricky to integrate, but have a look at my github repository and the README file explains how to do it. https://github.com/rbnpi/SPvisualiser

Give it a go and have some fun!

1 Like
10

I certainly will. Thanks so much @robin.newman

11

Spent a frustrating afternoon trying to get the visualiser to work with Raspberry Pi.
The sketch runs OK on the chromium browser on the Pi, but the issue is trying to get the html5 input to recognise the soundcard input from my pisound board. (I’m not using Sonic Pi at all initially) There’s probably some magical config I’m missing. I had better luck previsously using a straigth processing sketch.

So at the moment no joy with a Raspberry Pi.

UPDATE
I"ve managed to get sound input into the sketch running on teh Pi. However the sketch runs at about 67% capacity of CPU and so I don;t think there is any possibility of running Sonic Pi at the same time. However, I will investigate the possibility of running Sonic Pi on another machine, hard wiring the audio to the pi-sound card, and still using OSC to control the sketch form Sonic PI.