# Live Coding with Machine Learning (Magenta.js) **URL:** \ **Category:** Creations & Ideas\ **Created:** [October 5, 2020, 2:02am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462 "2020-10-05T02:02:40Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![mrbombmusic](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mrbombmusic/32/2282_2.png) [@mrbombmusic](https://in-thread.sonic-pi.net/u/mrbombmusic)\ **Post date:** [October 5, 2020, 2:02am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/1 "2020-10-05T02:02:40Z")
I made a project for a Hackathon last month where the theme was Music and AI (I didn’t win but I did place). It had to incorporate AI/Machine Learning into some type of musical project. I created a GUI that allows you to send drum patterns via OSC into Sonic Pi in real time as well as generate extensions of those drum patterns using a Machine Learning model called a Drum RNN (Recurrent Neural Network) which I got from the [Google Magenta Project](https://magenta.tensorflow.org/). You can also live code additional loops in Sonic Pi to accompany the drum patterns that are made in the GUI. I finally got around to making a video of a Live Coding session I did with it: [![](https://img.youtube.com/vi/Wi4WUCyiJ3w/hqdefault.jpg "Sonic Pi Drum RNN Live Coding session #1") ](https://www.youtube.com/watch?v=Wi4WUCyiJ3w) Here is a link to the Github repo I made: [GitHub - mrbombmusic/sonic-pi-drum-rnn-gui: A GUI that uses Magenta Drum RNN to create and generate drum patterns to send to Sonic Pi via OSC](https://github.com/mrbombmusic/sonic-pi-drum-rnn-gui) Here is a video I made which explains how it works: [Sonic Pi DrumRNN GUI - YouTube](https://youtu.be/9GHz_-xZPQs) It requires installing the [p5.js-OSC library](https://github.com/genekogan/p5js-osc) to make it work locally on your computer. I’m interested to see if others can get it up and running on their computers. Please let me know if you are able to get it working or have any questions. I would also be interested in getting this onto the web to make it more accessible for others to use. I know this would require some type of web based OSC library but I haven’t been able to find much about. If anyone is knows more about how to do that, I’d love to know more. Ideally, it would be great to have some type of function in Sonic Pi that could be used in tandem with this instead of just copying and pasting all the code needed to have it work, but that is just wishful thinking at this point. ---
**Author:** ![ethancrawford](https://avatars.discourse-cdn.com/v4/letter/e/a88e4f/32.png) [@ethancrawford](https://in-thread.sonic-pi.net/u/ethancrawford)\ **Post date:** [October 5, 2020, 2:10am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/2 "2020-10-05T02:10:59Z")
Looks like fun! I’d love to see if I can get this up and running soon, and have a play with it. I may not get to that for a little while though - pretty busy for the next few weeks 😅 (P.S. - Any chance you can add a license to your project? 🤞 Anyway, great work @mrbombmusic, looking forward to trying it out 🙂) ---
**Author:** ![mrbombmusic](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mrbombmusic/32/2282_2.png) [@mrbombmusic](https://in-thread.sonic-pi.net/u/mrbombmusic)\ **Post date:** [October 5, 2020, 2:39am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/3 "2020-10-05T02:39:44Z")
License added! Let me know how it goes. ---
**Author:** ![feline](https://avatars.discourse-cdn.com/v4/letter/f/ecd19e/32.png) [@feline](https://in-thread.sonic-pi.net/u/feline)\ **Post date:** [December 27, 2021, 6:21am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/4 "2021-12-27T06:21:18Z")
This is so excellent. But its not working for me: Runtime Error: [buffer 1, line 89] - NoMethodError Thread death ±-\> :live\_loop\_playDrumPatterns undefined method `[]' for nil:NilClass workspace_one:89:in `block (3 levels) in \_\_spider\_eval’ Please help. Thank you. ---
**Author:** ![ethancrawford](https://avatars.discourse-cdn.com/v4/letter/e/a88e4f/32.png) [@ethancrawford](https://in-thread.sonic-pi.net/u/ethancrawford)\ **Post date:** [December 27, 2021, 6:51am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/5 "2021-12-27T06:51:12Z")
@feline - what version of Sonic Pi are you using? ---
**Author:** ![feline](https://avatars.discourse-cdn.com/v4/letter/f/ecd19e/32.png) [@feline](https://in-thread.sonic-pi.net/u/feline)\ **Post date:** [December 27, 2021, 7:18am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/6 "2021-12-27T07:18:53Z")
3.3.1 on windows, Thanks ---
**Author:** ![ethancrawford](https://avatars.discourse-cdn.com/v4/letter/e/a88e4f/32.png) [@ethancrawford](https://in-thread.sonic-pi.net/u/ethancrawford)\ **Post date:** [December 28, 2021, 2:39am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/7 "2021-12-28T02:39:00Z")
Have done some brief tests and I can see the same result. No OSC messages appear to arrive in Sonic Pi, despite `setupOsc` referring to port 4560. @mrbombmusic - I notice in my browser console that I get a whole pile of CORS errors: ![image](https://europe1.discourse-cdn.com/flex017/uploads/sonic_pi/original/2X/5/545603273bf945dcb9c7f11118d4e5f6f47f9acc.png) Could this be why nothing is sent through? Not entirely sure, or how to fix it if so. ---
**Author:** ![feline](https://avatars.discourse-cdn.com/v4/letter/f/ecd19e/32.png) [@feline](https://in-thread.sonic-pi.net/u/feline)\ **Post date:** [December 28, 2021, 9:45pm UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/8 "2021-12-28T21:45:26Z")
Maybe it works on Mac, I tried Windows and Linux. Same CORS errors. Anyway, techno machine works and it was great fun to play with. The beauty is in not knowing what will happen, like fractals. Thanks, ---
**Author:** ![mrbombmusic](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mrbombmusic/32/2282_2.png) [@mrbombmusic](https://in-thread.sonic-pi.net/u/mrbombmusic)\ **Post date:** [December 29, 2021, 12:20am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/9 "2021-12-29T00:20:39Z")
Hi there. Sorry to see it isn’t working for you. The error message you are getting seems to be saying that there isn’t any drum pattern to play yet. The way it works is by sending the pattern made in the GUI grid to an array in Sonic Pi. I seem to recall getting similar errors when I would try running Sonic Pi prior to sending any drum patterns from the GUI. That would explain the ‘[]’ part at the beginning of the error message and why the error is coming from the playDrumPatterns live loop which is where the array for the drum pattern would play. I would recommend making a pattern in the GUI and sending it to Sonic Pi before you actually run the Sonic Pi code. You might need to try hitting run a few times in Sonic Pi. Also be aware that this was built in version 3.2.0 and I have not tried running it on any other versions. Someone else also raised an issue in the Github which might have also been related to using a different version. At some point, I will try to run it on 3.3 and see if it works for me. I also made it on a Mac and can not speak to how it might work on other operating systems. @ethancrawford I can’t speak to the CORS errors as I don’t even really know what that means. It is beyond my level of understanding. I didn’t originally make this project with the intention of it being used outside of my computer. Its great to see others trying to use it, but unfortunately I don’t have all the technical know how to trouble shoot why it may not work. If you are not seeing any OSC messages in Sonic Pi, I am wondering if it has something to do with the p5.js OSC library? ---
**Author:** ![ethancrawford](https://avatars.discourse-cdn.com/v4/letter/e/a88e4f/32.png) [@ethancrawford](https://in-thread.sonic-pi.net/u/ethancrawford)\ **Post date:** [December 29, 2021, 12:49am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/10 "2021-12-29T00:49:16Z")
Sure. I’m not terribly familiar with it either as you may have guessed 😂 my guess was also that it might be to do with the p5.js OSC library. _If_ that’s indeed the case, then if it or a different JS OSC library can be made to work, data might make it through to Sonic Pi 🙂 ---
**Author:** ![ethancrawford](https://avatars.discourse-cdn.com/v4/letter/e/a88e4f/32.png) [@ethancrawford](https://in-thread.sonic-pi.net/u/ethancrawford)\ **Post date:** [December 29, 2021, 5:55am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/11 "2021-12-29T05:55:20Z")
@mrbombmusic / @feline - see the following for a little fix which should hopefully prevent the errors from appearing when starting the Sonic Pi script. The only small procedure change in my update is that it would now necessary to start the Sonic Pi script first, before using the drum GUI. I tested this on v4 beta on Windows, but it should work the same in 3.3.1 too. Let me know what you think 🙂 > > > Previously, the script would generate several errors. > One seemed to be due to c…alling an invalid method on a hash. > Others seemed to be due to certain time state variables that sometimes > may have not been initialised properly. > These have now been fixed - the Sonic Pi script should (hopefully) no > longer generate errors when getting started. However, it also will not > play any sounds until at least both of the following are done > (in any order): > \- the user has sent the appropriate beats with the 'Send Beat' and/or > 'Send Generated Beat' buttons, depending on which type(s) of beat they > wish to play > \- the user has selected a drum kit with the kit dropdown selector. ---
**Author:** ![feline](https://avatars.discourse-cdn.com/v4/letter/f/ecd19e/32.png) [@feline](https://in-thread.sonic-pi.net/u/feline)\ **Post date:** [December 29, 2021, 7:11am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/12 "2021-12-29T07:11:50Z")
Sonic pi receiver.rb no errors, and console in Chrome : polling-xhr.js:157 GET [http://127.0.0.1:8081/socket.io/?EIO=4&transport=polling&t=Nu50l70](http://127.0.0.1:8081/socket.io/?EIO=4&transport=polling&t=Nu50l70) net::ERR\_CONNECTION\_REFUSED and node bridje.js follows with node:dgram:917 throw new ERR\_SOCKET\_DGRAM\_NOT\_RUNNING(); It must be P5js-osc issue. ---
**Author:** ![ethancrawford](https://avatars.discourse-cdn.com/v4/letter/e/a88e4f/32.png) [@ethancrawford](https://in-thread.sonic-pi.net/u/ethancrawford)\ **Post date:** [December 29, 2021, 1:49pm UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/13 "2021-12-29T13:49:23Z")
How are you starting your web server/accessing the web GUI? I’ve been able to get things working on Windows and macOS (with my proposed patch that I linked above) by doing the following: - starting the node bridge according to the instructions at [GitHub - genekogan/p5js-osc: OSC for p5.js with examples](https://github.com/genekogan/p5js-osc) - starting a python web server similarly to how someone did it as described in [Sending OSC to Sonic Pi. · Issue #2 · mrbombmusic/sonic-pi-drum-rnn-gui · GitHub](https://github.com/mrbombmusic/sonic-pi-drum-rnn-gui/issues/2) - loading the web GUI in a browser at `http://localhost:8000/GUI Code/` - making sure that on the following line, the port number used to send OSC outwards from script.js was edited to match the number labeled `Incoming OSC port` in the Sonic Pi IO settings (on my macOS machine it was different): > (In my case, the incoming port that Sonic Pi reported in the preferences panel was not 4560, so I changed that from 4560 to the value it _did_ say, in the line shown above in the script.js file). - Starting the Sonic Pi receiver script - Creating beats in the web GUI, pressing send beat, and selecting a kit …and I had sound. ---
**Author:** ![feline](https://avatars.discourse-cdn.com/v4/letter/f/ecd19e/32.png) [@feline](https://in-thread.sonic-pi.net/u/feline)\ **Post date:** [December 30, 2021, 9:55am UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/14 "2021-12-30T09:55:38Z")
Yes it works. It was npm install in p5js-osc. It said to do force audit, I complied and that was a mistake. Once I left it alone, it all worked. Never give up. Thank you Ethan. And now the music. Happy New Year! ---
**Author:** ![ethancrawford](https://avatars.discourse-cdn.com/v4/letter/e/a88e4f/32.png) [@ethancrawford](https://in-thread.sonic-pi.net/u/ethancrawford)\ **Post date:** [December 30, 2021, 3:01pm UTC](https://in-thread.sonic-pi.net/t/live-coding-with-machine-learning-magenta-js/4462/15 "2021-12-30T15:01:10Z")
Great to hear you got it working! Happy new year to you also 😄