Spring clean to enable contributions


#1

After building Sonic Pi on my own workstation, I’ve spent the last few days thinking how to make it easier for others to make contributions to the project. As Sam has said on this forum, he has been facing a resource shortage for the last year or so, where his limited time is split between either working on Sonic Pi or doing workshops, talks and performances to raise awareness of the project and put beans on the table. This leaves no time for improving the developer experience and managing the project - with the result that new contributors face a steep learning curve, and existing contributors may lose motivation or become frustrated.

I would like to make it easier to access Sonic Pi as a potential contributor, in two ways: firstly, to make sure all platforms are served with up-to-date packages. 3.1 is currently packaged for Window and Mac, but Raspbian lags at 2.10, and desktop Linux is officially unsupported, with a few outdated packages scattered across third-party repositories. As a long-term Linux developer, I am certainly biased, but I believe that Linux users often become contributors, and the gateway is an available package to become a user.

Secondly, I want to make it easy to build and hence hack on Sonic Pi. Fearsome dependencies have to be tamed and built first, as it’s based on Supercollider, Ruby, Qt and Erlang. The codebase is currently something of a rabbit warren, with many deeply branching tunnels, chambers of last season’s carrots, and communicating cross-passages. Working on one component requires bunny lore in case it disturbs a support somewhere else and causes a collapse. There is no single build system, instead, there is a sheaf of platform/distribution/version-specific scripts provided by plucky individuals, all doing the same thing, if they are not stale. There is no install target, so packagers have to figure out what goes where, or just hold their noses and bang the whole tree into /opt.

Isn’t this a fairly mordant first post to the Development category? What is he going to do about it, you ask…


#2
  1. The first thing the project needs is a simpler, flatter codebase with less coupling between components.

  2. This enables the creation of a cross-platform build system, which separates the installation of Sonic Pi’s dependencies from its build

  3. This will simplify correct packaging of the build, and enable curious hackers to clone the repository and build Sonic-Pi locally and maybe experiment with changing the code. It should also save existing contributors’ time and effort.

Drilling down into 1), I intend to:

  • Remove unused components (html client) and cruft
  • Squash the tree by function of each component
  • Identify shared stuff and push that up the tree, removing coupling between components
  • Separate build utilities from the server code
  • Use Bundler to manage gems instead of keeping vendor copies in the tree

Following the motto “Move fast and break things” there is a first stab at this in my fork. It probably breaks a lot of things including Mac and Windows for now, but I will loop back round once I have a CMake build system up.


#3

Awesome!

Not sure if you’ve already seen it, and not sure how relevant it still is, but there was a PR opened a few years ago which generated some discussion about using bundler.


#4

I hadn’t seen that (probably for the best), but it looks like the discussion in that (and the linked Linux Packaging PR #1864) mention many of the problems I’m trying to solve. I worked on a packaged Ruby CLI application recently, and the approach we took on Linux was to run bundler in --standalone mode during the package build, setting up the bundled gems so no bundler is required at runtime. For Windows and Mac builds, I would insert ‘grab Ruby binaries and bundler’ and run bundle install using those, before packaging them and the gems, deleting specs, examples and other ancillary stuff as I do so.


#5

CMake build system is up, finds dependencies, builds the scheduler, bundles for the server, builds the gui: https://github.com/wstephenson/sonic-pi/tree/cmake-all-the-things
Still to-do: doc and i18n, Windows and Mac support.

Now I’ll dare to go check whether there’s an existing cmake PR…


#6

Having the building and packaging of Sonic Pi simplified is something that I’m very, very much in favour of. The same with the codebase in general (particularly if we can somehow even introduce some level of good coding style with linting etc - though that bit is just my own preference!)

My knowledge is fairly limited when it comes to packaging/building systems etc, but if there’s anything I can do to assist you while you continue your good work @wstephenson, let me know.


#7

Been working on recruiting someone that will hopefully be able to help out. It’s a bit of a tit for tat thing for me to get a talented developer. He was describing a controller plugin and talking about how the concept had never been seen. What he was describing was basically some techniques that I’ve come to rely on Sonic Pi for. I told him that rather than reinventing that wheel, I’d rather he put his focus into a GUI that can link back to what SPi can already do extremely well, schedule events in virtual time. I told him that I’d like for Sonic Pi to be at the point where it’s easy to build different versions, for instance a MIDI-only build, a streamlined CLI, a portable library, etc. He liked that idea, and I got a call from him Tuesday saying he’d been curious and looked into the github but was locked out. I told him about the Patreon and the current state of things, but given the reason for all this, I want to let him use my working UbuntuStudio build of 3.2 to see where things are. ( @samaaron Of course I’d like to get your blessing first since the dev builds are somewhat restricted)

Rand is a talented professional programmer with a passion for electronic music, especially PsyTrance. Contributing to a project that will allow for creative new techniques is something that he’d to for the love of it, as he’s told me already in regards to SPi. He currently works for PixelJam games on a sequel to their Dino Run game, he’s been overhauling and porting the original engine and letting us try out concept builds for a few months now. He introduced me to the PixelJam crew, and they’re intrigued by the idea I mentioned of having a lightweight music system that can create complex and reactive chiptune pieces. Rand has told me to keep a contribution in mind for his game. Additionally he’s brilliant with shader code and enables some wild reactive visualizations on LED screens at some of our temperate rainforest dance gatherings. Dude knows his stuff.

So let me know how you feel about me trying to invite someone into the development community who isn’t otherwise currently involved with Sonic Pi, I’d be really keen to do anything I can to keep it under active development. It allows me to do things easily that would take me way too long to work out in Purr-Data or SuperCollider itself (trust me I tried in between 3.2 build attempts :smiley: ) .


#8

Thanks for talking to other developers about helping out with Sonic Pi.

Right now, I’m very interested in patches that help make Sonic Pi easier to package and maintain on Linux, but it also needs to not interfere with the ease of packaging and maintaining things on macOS and Windows too.

For much bigger, structural changes (headless, MIDI-only, etc.) this would definitely need to be discussed beforehand. Whilst I would love Sonic Pi to go into many different directions, ultimately any code that goes into the codebase needs to be code that I can maintain when the original developer moves onto pastures new.

:slight_smile:


#9

I’m totally on board with that, I’m excited to be able to teach sonic pi no matter what platform my schools use. We chatted some more at a show I played on Saturday, I did mention the specific topics being addressed in this thread, that first priority is having a rock solid installer and maintenance volunteers for each platform (I think some of your Linux guys are chomping at the bit and Rand is a Windows-first person ) that allow them to stay on schedule with each other and as equal releases. Second priority in my mind is to have a big refactoring party to help make the code easier to maintain and make sure everything on the Ruby side is structured DRY and fully modular. I think once we can approach various subsystems as discreet modules, then it will be easier to make alt versions of SPi as features that can be optionally added at runtime or hard built beforehand to reduce dependencies and shrink builds.