After building Sonic Pi on my own workstation, I’ve spent the last few days thinking how to make it easier for others to make contributions to the project. As Sam has said on this forum, he has been facing a resource shortage for the last year or so, where his limited time is split between either working on Sonic Pi or doing workshops, talks and performances to raise awareness of the project and put beans on the table. This leaves no time for improving the developer experience and managing the project - with the result that new contributors face a steep learning curve, and existing contributors may lose motivation or become frustrated.
I would like to make it easier to access Sonic Pi as a potential contributor, in two ways: firstly, to make sure all platforms are served with up-to-date packages. 3.1 is currently packaged for Window and Mac, but Raspbian lags at 2.10, and desktop Linux is officially unsupported, with a few outdated packages scattered across third-party repositories. As a long-term Linux developer, I am certainly biased, but I believe that Linux users often become contributors, and the gateway is an available package to become a user.
Secondly, I want to make it easy to build and hence hack on Sonic Pi. Fearsome dependencies have to be tamed and built first, as it’s based on Supercollider, Ruby, Qt and Erlang. The codebase is currently something of a rabbit warren, with many deeply branching tunnels, chambers of last season’s carrots, and communicating cross-passages. Working on one component requires bunny lore in case it disturbs a support somewhere else and causes a collapse. There is no single build system, instead, there is a sheaf of platform/distribution/version-specific scripts provided by plucky individuals, all doing the same thing, if they are not stale. There is no install target, so packagers have to figure out what goes where, or just hold their noses and bang the whole tree into /opt.
Isn’t this a fairly mordant first post to the Development category? What is he going to do about it, you ask…
2 Likes
-
The first thing the project needs is a simpler, flatter codebase with less coupling between components.
-
This enables the creation of a cross-platform build system, which separates the installation of Sonic Pi’s dependencies from its build
-
This will simplify correct packaging of the build, and enable curious hackers to clone the repository and build Sonic-Pi locally and maybe experiment with changing the code. It should also save existing contributors’ time and effort.
Drilling down into 1), I intend to:
- Remove unused components (html client) and cruft
- Squash the tree by function of each component
- Identify shared stuff and push that up the tree, removing coupling between components
- Separate build utilities from the server code
- Use Bundler to manage gems instead of keeping vendor copies in the tree
Following the motto “Move fast and break things” there is a first stab at this in my fork. It probably breaks a lot of things including Mac and Windows for now, but I will loop back round once I have a CMake build system up.
5 Likes
Awesome!
Not sure if you’ve already seen it, and not sure how relevant it still is, but there was a PR opened a few years ago which generated some discussion about using bundler.
I hadn’t seen that (probably for the best), but it looks like the discussion in that (and the linked Linux Packaging PR #1864) mention many of the problems I’m trying to solve. I worked on a packaged Ruby CLI application recently, and the approach we took on Linux was to run bundler in --standalone mode during the package build, setting up the bundled gems so no bundler is required at runtime. For Windows and Mac builds, I would insert ‘grab Ruby binaries and bundler’ and run bundle install using those, before packaging them and the gems, deleting specs, examples and other ancillary stuff as I do so.
3 Likes
CMake build system is up, finds dependencies, builds the scheduler, bundles for the server, builds the gui: https://github.com/wstephenson/sonic-pi/tree/cmake-all-the-things
Still to-do: doc and i18n, Windows and Mac support.
Now I’ll dare to go check whether there’s an existing cmake PR…
3 Likes
Having the building and packaging of Sonic Pi simplified is something that I’m very, very much in favour of. The same with the codebase in general (particularly if we can somehow even introduce some level of good coding style with linting etc - though that bit is just my own preference!)
My knowledge is fairly limited when it comes to packaging/building systems etc, but if there’s anything I can do to assist you while you continue your good work @wstephenson, let me know.
2 Likes
Been working on recruiting someone that will hopefully be able to help out. It’s a bit of a tit for tat thing for me to get a talented developer. He was describing a controller plugin and talking about how the concept had never been seen. What he was describing was basically some techniques that I’ve come to rely on Sonic Pi for. I told him that rather than reinventing that wheel, I’d rather he put his focus into a GUI that can link back to what SPi can already do extremely well, schedule events in virtual time. I told him that I’d like for Sonic Pi to be at the point where it’s easy to build different versions, for instance a MIDI-only build, a streamlined CLI, a portable library, etc. He liked that idea, and I got a call from him Tuesday saying he’d been curious and looked into the github but was locked out. I told him about the Patreon and the current state of things, but given the reason for all this, I want to let him use my working UbuntuStudio build of 3.2 to see where things are. ( @samaaron Of course I’d like to get your blessing first since the dev builds are somewhat restricted)
Rand is a talented professional programmer with a passion for electronic music, especially PsyTrance. Contributing to a project that will allow for creative new techniques is something that he’d to for the love of it, as he’s told me already in regards to SPi. He currently works for PixelJam games on a sequel to their Dino Run game, he’s been overhauling and porting the original engine and letting us try out concept builds for a few months now. He introduced me to the PixelJam crew, and they’re intrigued by the idea I mentioned of having a lightweight music system that can create complex and reactive chiptune pieces. Rand has told me to keep a contribution in mind for his game. Additionally he’s brilliant with shader code and enables some wild reactive visualizations on LED screens at some of our temperate rainforest dance gatherings. Dude knows his stuff.
So let me know how you feel about me trying to invite someone into the development community who isn’t otherwise currently involved with Sonic Pi, I’d be really keen to do anything I can to keep it under active development. It allows me to do things easily that would take me way too long to work out in Purr-Data or SuperCollider itself (trust me I tried in between 3.2 build attempts 
) .
1 Like
Thanks for talking to other developers about helping out with Sonic Pi.
Right now, I’m very interested in patches that help make Sonic Pi easier to package and maintain on Linux, but it also needs to not interfere with the ease of packaging and maintaining things on macOS and Windows too.
For much bigger, structural changes (headless, MIDI-only, etc.) this would definitely need to be discussed beforehand. Whilst I would love Sonic Pi to go into many different directions, ultimately any code that goes into the codebase needs to be code that I can maintain when the original developer moves onto pastures new.
1 Like
I’m totally on board with that, I’m excited to be able to teach sonic pi no matter what platform my schools use. We chatted some more at a show I played on Saturday, I did mention the specific topics being addressed in this thread, that first priority is having a rock solid installer and maintenance volunteers for each platform (I think some of your Linux guys are chomping at the bit and Rand is a Windows-first person ) that allow them to stay on schedule with each other and as equal releases. Second priority in my mind is to have a big refactoring party to help make the code easier to maintain and make sure everything on the Ruby side is structured DRY and fully modular. I think once we can approach various subsystems as discreet modules, then it will be easier to make alt versions of SPi as features that can be optionally added at runtime or hard built beforehand to reduce dependencies and shrink builds.
1 Like
Hello all.
Is there any chance we can revive these efforts?
I’m personally finding it quite limiting that I’m unable to develop Sonic Pi on multiple platforms.
I’ll admit that it certainly doesn’t help when I come from an (albeit full-stack) web development background, and compiling native apps isn’t my strong point. Despite this, I know enough to (usually) manage to get the macOS app building and running successfully. However, I have yet to do the same for Windows. (For example, I’ve managed just recently to get my Windows build compiling, but have no idea just yet why it seems to be unable to get past the point of talking to SuperCollider without crashing).
If the two steps of simplifying the code base and automating the build process can work towards eliminating problems like this, then I’m really keen for them to happen. This however would probably benefit from someone (or multiple people 
) who is/are more experienced with native app development than I currently am, being able to assist me/us.
If any of you folks who have expressed interest in this idea think you might be able to jump back in, or know others who can help, please let us know - I’d really love to make things easier to build and maintain.
Yes it would be good to get further work on making it easier to build Sonic Pi on different platforms. Understandably Sam does not have the resources to develop build/install mechanisms for many different OS, and indeed packaging is a completely different ball game to just building the app.
This is why currently Sonic Pi is somewhat languishing on Raspbian. The introduction of Raspbian Buster necessary to support the new Pi4 meant that the previous package for Raspbian Stretch no longer worked and the only deb package available was the one for Sonic PI 2.10. This has now been upgraded (with no input from Sam) to a so called version 3.1, but unfortunately the build is incomplete, lacking erlang and osmid support and so it does NOT support midi or OSC messaging, despite having all the documentation about these. This very much concerns me, and I think it will be detrimental to many new users really getting to grips with Sonic Pi.
I have produced instructions for build from source for version 3.1 (which incidentally was never released for the Raspberry Pi, the previous version being 3.0.1). I have made 4 corrections to the tagged 3.1 source to enable it to build on a more recent OS than when it was released (Jan 2018 for Mac), and to correct one bug which prevented it from stopping erlang and osmid on quitting, and the whole build can be completed from scratch in well under an hour.
I have also produced a stripped down folder of Sonic Pi which can be installed with a simple script in a matter of minutes on any SD card with the latest Raspbian Buster 2019-09-26. It can coexist with the supplied cutdown Sonic Pi (listed as Sonic Pi in the menu) and it installs a menu entry entitled Sonic Pi 3.1
I have a similar script and package for Sonic Pi 3.2dev although for obvious reasons this is still changing!
I haven’t released the 3.1 details yet, but with a proper .deb package not looking likely to appear in the near future it may be useful to do so. What do you think?
As far as MacOS goes I build this fairly frequently, but I have found one or two pitfalls currently I use High Sierra still although I have tried the build on a virtual Mohave. The main problem has been keeping the same version of Boost Libraries for SuperCollider and for the Scope code. Now I use a self build SuperCollider and then link to the boost libraries contained in it. Package building is probably easier for Mac but Catalina is introducing the need for much more stringent signing and may cause problems.
I’m afraid I gave up with Windows builds, as I don’t like the platform, and I only have a virtual PC.
Anyway, it would be good to have any experienced package builders on board, although Sonic Pi is not the simplest to package because of its disparate components: Ruby, Supercollider, Qt, osmid, erlang.
1 Like
I managed earlier today to get my Windows build compiling and running 
I still hope someone can assist us with simplifying the codebase/improving the build process though. @wstephenson and/or @kniknoo - any news? Making things more cross-platform would be a huge improvement - Maybe we could build on Will’s initial work as linked above? 
1 Like