Tokenizationin in sonic pi

Is there a construct in Sonic pi in which you can tokenize a string? I’ve tried .includes and .split that are part of Ruby, but they didn’t work in Sonic Pi. Thanks for your help

There’s no such constructs in Sonic Pi language, but Ruby works just fine … at least for now:

test = "1 2 3 4"

splitted = test.split(" ")

print splitted

There’s a lot of examples for text sonification etc. in the forum. You can also use Treetop, which is included as a requirement by default:

Treetop.load_from_string("
grammar Foo

  rule my_pattern
   (integer / space)+
  end

 rule integer
   [0-9]+
  end

  rule space
    [\S\n]+
  end
end
")

parser = FooParser.new
result = parser.parse("1 2 3 4")

if !result # Some error handling
  puts parser.failure_reason
  puts parser.failure_line
  puts parser.failure_column
end

print result

But all of this is of course, not Sonic Pi language and may break in the future.

1 Like

I would love to understand more how Treetop works. I wrote an Request For Comment on adding tabulature notation to Sonic Pi and would love comments/feedback/collaborators on it

The simple answer to this is that Sonic Pi doesn’t currently support any tokenisation at this point.

However, building internal DSLs and mini pattern languages/notations is something I’d like to see people explore - it’s just that that exploration should be considered off-piste and nothing is guaranteed to work or to continue to work.

Ultimately I’d like to provide useful tooling for this, but it’s quite far down the roadmap for me personally - especially as I’m looking hard towards moving a lot of the core language from Ruby to Elixir/Erlang which essentially means rewriting it from the ground up. Therefore, any efforts put into extending the Ruby syntax should purely be seen as experimental and as a way to explore what works and what doesn’t from a design perspective.

Having a few mini languages that people have written and work with will be very useful to help guide the future design of more official support for this stuff.

1 Like

Now that was an interesting post. I’m also fond of minimalism and been doing a lot of unguaranteed experimentation in Sonic Pi over the years. If you want examples how to use treetop check out ziffers parser.

Ziffers is all about generative and numeric notation, but it can also be used as tab-like notation of sorts for samples:

kick          = tabb(["1000","1000","1000","1000","1000"])
snare         = tabb(["0100","0100","0100","0100","0100"])
closed_hi_hat = tabb(["1101","1101","1101","1101","1100"])
open_hi_hat   = tabb(["0010","0010","0010","0010","0011"])

-to ziffers->

beat = "[: KC SC O C :4] KC SC O O"

To run in Sonic Pi require Ziffers and play:

samples = {
  "K": :drum_bass_hard,
  "S": :drum_snare_hard,
  "C": :drum_cymbal_closed,
  "O": :drum_cymbal_open
}

beat = "[: KC SC O C :4] KC SC O O"

z1 beat, use: samples, sleep: 0.25 # Loop it

Instead of defining default sleep you could alternatively use alternative note duration notations:

beat2 = "[: q KC SC O C :3] h K K" # char durations
alt_beat = "[: 0.125 KC SC O C :3] 0.25 K K" # same as numeric durations
alt_beat = "[: [KC SC [C C] [O O]] :2] [: [KC SC [O O] [C C]] :2]" # Tidalish subdivision
1 Like

Oooh, lovely will have a play, but probably not until I have done the live coding at the Festival in Berlin that made me look into this in the first place :wink:

1 Like

ha-ha, good thing my specialism is in writing programming languages for the BEAM and I haven’t used Ruby in a serious way for 20 years :wink:

1 Like

Once you recognise that Regex’s are a separate programming language embedded into most other programming languages then the problem of Ruby/Elixir goes away - tabulatures are Just Another Mini Programming Language. The syntax can be implemented in either language. In fact Elixir with its sigils has the perfect mechanism to drop in tabulature language.

1 Like

And APL man … :exploding_head:. Some live repl for APL with MIDI out support would be so dope.

Got a REPL called Rappel for Pometo (bit off topic here tho)