Live Coding with Machine Learning (Magenta.js)

I made a project for a Hackathon last month where the theme was Music and AI (I didn’t win but I did place). It had to incorporate AI/Machine Learning into some type of musical project. I created a GUI that allows you to send drum patterns via OSC into Sonic Pi in real time as well as generate extensions of those drum patterns using a Machine Learning model called a Drum RNN (Recurrent Neural Network) which I got from the Google Magenta Project. You can also live code additional loops in Sonic Pi to accompany the drum patterns that are made in the GUI.

I finally got around to making a video of a Live Coding session I did with it:

Here is a link to the Github repo I made: https://github.com/mrbombmusic/sonic-pi-drum-rnn-gui
Here is a video I made which explains how it works: https://youtu.be/9GHz_-xZPQs

It requires installing the p5.js-OSC library to make it work locally on your computer.

I’m interested to see if others can get it up and running on their computers. Please let me know if you are able to get it working or have any questions.

I would also be interested in getting this onto the web to make it more accessible for others to use. I know this would require some type of web based OSC library but I haven’t been able to find much about. If anyone is knows more about how to do that, I’d love to know more. Ideally, it would be great to have some type of function in Sonic Pi that could be used in tandem with this instead of just copying and pasting all the code needed to have it work, but that is just wishful thinking at this point.

3 Likes

Looks like fun!
I’d love to see if I can get this up and running soon, and have a play with it. I may not get to that for a little while though - pretty busy for the next few weeks :sweat_smile:

(P.S. - Any chance you can add a license to your project? :crossed_fingers: Anyway, great work @mrbombmusic, looking forward to trying it out :slight_smile:)

1 Like

License added!

Let me know how it goes.

2 Likes