Sonic Pi x p5js

Hey all - I recently became a patreon supporter and have the goal of trying to create a cross media experience where both sound and visuals can be live coded, all while performing as an avatar. I’m trying to get p5js to talk to Sonic Pi via OSC and have tried every github repo that seems to be in existence on the topic, but am still at a loss. I’m relatively new to both js and sonic pi, but have been able to get OSC signals coming out of my ruby code and into TouchOSC - just not as far as getting it into a locally served p5js web browser instance. If anyone has any tips or tricks on how to accomplish this, they would be so, so appreciated as I’m really grasping at straws.

Thank you, and thanks for having me in this community!

2 Likes

Ha! Just this minute, I was coming back to this forum thinking about p5js! (After hearing Sam’s interview on the Elixir Newbie podcast, I’ve been thinking about other languages/environments for creative endeavours. I really like p5js and it has some useful libraries for sound and music.)

So, it sounds like you’ve identified the obstacle, which is between the two environments, not easily addressed within each.
The key might be in trying other things with OSC. Typically, there’s a bit of configuration that you need to do. Possibly converting some of the messages so that they make sense on the other side.
At the same time, there are people here who’ve played enough with OSC on the SPi side who could help you more specifically.
What are the messages you want to send, in which direction? You want p5js to trigger things in SPi or the reverse?

Amazing! I’ve done a little bit of work with OSC (filtering, forwarding, routing etc.) in other programs, but I’m wanting to send OSC data from SPi to p5js. For example, if I had a cuttoff and amp value in SPi that was rrand(0,100), I would want that cutoff and amp value to be sent to p5js as two different addresses/args (/cutoff, /amp) to control the fill alpha and ellipse size. I have SPi working in sending values to TouchOSC to do something similar, it’s more on the p5 front that I can’t figure out how to pull in this data. Thanks for the quick response!

Can’t remember how p5js deals with OSC. Maybe it’s through an external library?
Like GitHub - genekogan/p5js-osc: OSC for p5.js with examples ?
Going back to Andreas Schlegel - oscP5

There’s one part of the issue which is about the connection itself (OSC is basically a networking protocol). Another about how those messages are interpreted.
The p5js crowd tends to be very welcoming. Have you asked them about getting OSC input?

Hi @itsmetamike,

I have had success working with using OSC between p5 and Sonic Pi. Here are a couple examples of sketches I made to visualize some function of Sonic Pi.
Onset option: Sonic Pi - Onset parameter visualization using p5.js - YouTube
Spread Function: Sonic Pi - Spread function visualization w/ p5.js - YouTube

There are several steps to get started but its pretty easy once you get things up and running.

I recommend using Gene Kogan’s p5js-osc library: GitHub - genekogan/p5js-osc: OSC for p5.js with examples

You will need to have node.js installed: Download | Node.js

After installing node, you should clone the p5js-osc repo as instructed on the github page (you will need to have some basic knowledge of getting around in the terminal to do this):

$ git clone https://github.com/genekogan/p5js-osc
$ cd p5js-osc/
$ npm install

Once these things have been installed, here is how you can proceed each time you want to have Sonic Pi and P5 communicate.

You will need to open a terminal window and start a local server.
Here are some different possibilities for how you can do this: Local server · processing/p5.js Wiki · GitHub

If you have a mac, I think the python simple server is one of the easier ways to go.
Just write this in the terminal:

python -m SimpleHTTPServer

Once you have this up and running, you need to open another terminal window and navigate to the p5js-osc directory using the ‘cd’ command in your terminal (‘cd’ stands for ‘change directory’)

Once you are in the p5js-osc directory, you need to run the following command:

node bridge.js

Then open your web browser and go to: http://localhost:8000

This will bring up the directories of all the folders on your computer.
Navigate your way to the p5js-osc folder. Then open the file you want to use.

If you are trying to send messages from Sonic Pi to p5js-osc, I would recommend starting with the p5-basic example provided in the repository. I made some modifications below to fit what you had mentioned. This example will look for two OSC messages called “/cutoff” and “/amp” and when it receives those messages, the cutoff message value will affect the alpha of the ellipse and the amp message will affect the size (although I would not recommend using amp if you plan on using values higher than double digits, the sound will get really loud!)

var ellipseAlpha = 100 // initialized variable for alpha
var  ellipseSize = 50; // initialized variable for size

function setup() {
	createCanvas(500, 500);
	setupOsc(3333, 4560); // OSC ports. you'll need 3333 in your Sonic Pi code
}

function draw() {
	background(0, 0, 255);
	fill(0, 255, 0, ellipseAlpha); // fill using alpha variable
	ellipse(200, 200, ellipseSize, ellipseSize); // ellipses function using size variable

}

// this function controls the incoming OSC messages
function receiveOsc(address, value) {
	console.log("received OSC: " + address + ", " + value);

// This conditional statement looks for "cutoff" message and assigns its value to alpha variable
	if (address == '/cutoff') { 
		ellipseAlpha= value[0];
	}
// This conditional statement looks for "amp" message and assigns its value to size variable
if (address == '/amp') {
		ellipseSize = value[0];
	}
}

// This function is for sending OSC messages
// This is not relevant for receiving OSC messages
function sendOsc(address, value) {
	socket.emit('message', [address].concat(value));
}

function setupOsc(oscPortIn, oscPortOut) {
	var socket = io.connect('http://127.0.0.1:8081', { port: 8081, rememberTransport: false });
	socket.on('connect', function() {
		socket.emit('config', {
			server: { port: oscPortIn,  host: '127.0.0.1'},
			client: { port: oscPortOut, host: '127.0.0.1'}
		});
	});
	socket.on('message', function(msg) {
		if (msg[0] == '#bundle') {
			for (var i=2; i<msg.length; i++) {
				receiveOsc(msg[i][0], msg[i].splice(1));
			}
		} else {
			receiveOsc(msg[0], msg.splice(1));
		}
	});
}

On the Sonic Pi side, you can just run this code to send osc messages using those message names that p5 is looking for:

use_osc "localhost", 3333 # This value matches the port value in the setupOsc function in the p5 sketch
live_loop :oscTest do
  osc "/cutoff" , rrand(1, 100) # modify this value however you want
  osc "/amp", rrand(1, 200) # modify this value however you want
  sleep 1
end

This code doesn’t include anything music related but you can add that easy enough. However, if you are going to use random values, I would recommend storing those values in variables so they will be consistent when used for both OSC messages and Sonic Pi parameters.

Example:

use_osc "localhost", 3333
live_loop :oscTest do
 n1 = rrand(1, 100) 
  osc "/cutoff" , n1 
  play 60, cutoff: n1
  sleep 1
end

I hope this is useful for you. I know it is a lot to work through but once you understand it, it is pretty easy to work with. I am going to try to put together a video tutorial to go over this process sometime in the near future, but hopefully this gets you to where you want to go. Let me know if you have any questions.

Good Luck!

3 Likes

I was scouring your code until about 4am this morning trying to do a similar process from your various p5 editor scripts and other github repos but having you write it out step by will (fingers crossed) hopefully give me what I’m looking for. Thank you so much! Is there somewhere that I can follow your work?

1 Like

I have asked the p5 discord but I’m hoping that @mrbombmusic’s comment will be able to help me solve for this. Will let you know tomorrow!

Most of my stuff I post on my youtube channel: youtube.com/mrbombmusic

I have twitter: @mrbombmusic But I haven’t posted there much recently but you can scroll through my past posts and see if anything interests you.

Hope it works for you. Just FYI, I don’t think you can send OSC messages through the p5 Web Editor. It needs to be done locally, hence setting up the local server.

2 Likes

It pains me to say it but I’ve spent another few hours on this tonight and am at an absolute loss. I just cant set up the server correctly for Windows. Tried node, python and xcode but nothing is working and i keep getting errors. will need to dig up an old mac tomorrow to see if i can get it working on there. thanks so much for the help though!

Sorry to hear that. I don’t have any experience with doing this stuff on Windows so unfortunately I can’t be much help on that end. I will try and play around with one of my laptops at school to see if I can get it working and I’ll get back to you. Keep me updated on your progress.

1 Like