# First Sonic Pi track, made with Literate Programming

**URL:** <https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308>\
**Category:** Creations & Ideas\
**Created:** [March 25, 2021, 9:49pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308 "2021-03-25T21:49:28Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [March 25, 2021, 9:49pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/1 "2021-03-25T21:49:29Z")

</div>

Hi there!

Here is my first track made with Sonic Pi, and also my first real piece of music at all.

[House Experiment 001 - Sonic Pi](https://www.thefuture.fm/track/862/house-experiment-001-sonic-pi "House Experiment 001 - Sonic Pi")

The track was written using Literate Programming. This means there is a textual description of the project, including notes/scores, which contains the complete code of the track:

[https://mlange-42.github.io/Sonic-LP/first-steps/house-bounce.html](https://mlange-42.github.io/Sonic-LP/first-steps/house-bounce.html)

Using a command line tool, the code is extracted from the documentation into a code file usable with Sonic Pi.

Please bear with me if I used quite some nonsense terminology in the descriptions, or made mistakes in the scores. I have literaly no idea about musical theory, except that I know how notes of different length look like. 😉 So, I would be happy about any hints on false terms used there.

The extracted code is available on GitHub:

> <https://github.com/mlange-42/Sonic-LP/blob/main/code/House/HouseBounce.rb>

EDIT: Is it possible to embed an external track from TheFuture.fm here?

---

<div class="post-metadata">

**Author:** ![d0lfyn](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/d0lfyn/32/2254_2.png) [@d0lfyn](https://in-thread.sonic-pi.net/u/d0lfyn)\
**Post date:** [March 25, 2021, 10:17pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/2 "2021-03-25T22:17:15Z")

</div>

OMG this is such a cool project! I didn’t realise tools like these existed. I love how you’re presenting your ideas, and I hope to hear more from you.

Your descriptions look fine at first glance 👍

---

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [March 25, 2021, 10:29pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/3 "2021-03-25T22:29:51Z")

</div>

Many thanks, @d0lfyn !

Literate Programming has a relatively long tradition, but certainly in a narrow niche. E.g., TeX was written this way, starting in 1981.

Due to the fact that I found no such tool that is easy to use and uses a modern and simple syntax (here: Markdown), I startet [Yarner](https://github.com/mlange-42/yarner) one year ago.

---

<div class="post-metadata">

**Author:** ![d0lfyn](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/d0lfyn/32/2254_2.png) [@d0lfyn](https://in-thread.sonic-pi.net/u/d0lfyn)\
**Post date:** [March 25, 2021, 10:54pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/4 "2021-03-25T22:54:26Z")

</div>

I’m certainly interested to see what becomes of Yarner and this approach to programming!

---

<div class="post-metadata">

**Author:** ![Nechoj](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/nechoj/32/1814_2.png) [@Nechoj](https://in-thread.sonic-pi.net/u/Nechoj)\
**Post date:** [March 25, 2021, 11:04pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/5 "2021-03-25T23:04:11Z")

</div>

Great stuff! Yes, literal programming was invented by Donald Knuth, because he wanted code and documentation of the code to be the same thing. But TeX was so extremely well done that it became a tool for creating high quality documents on its own, without the code. Most scientific publications are written in TeX (or LaTeX) nowadays.

A similar aproach was taken with JuPyter notebooks, a programming and documentation format that tightly integrates code and documentation (markdown, HTML and LaTex). JuPyter notebooks already integrate with the runtime environment, so that you can run the code directly from the intercative notebook without actually extracting the code.

Your solution Yarner is more in the spirit of TeX with marking up the code parts and then extract those. Again, super interesting stuff, congrats! And quite a complex piece of music as a starter 👍

---

<div class="post-metadata">

**Author:** ![nlb](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/nlb/32/997_2.png) [@nlb](https://in-thread.sonic-pi.net/u/nlb)\
**Post date:** [March 26, 2021, 8:52am UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/6 "2021-03-26T08:52:04Z")

</div>

Hi,

Thanks for sharing your tool with the sonic pi community

> [@mlange](#):
>
> Using a command line tool, the code is extracted from the documentation into a code file usable with Sonic Pi.

Could you explain the command line to create the output ? How to install the yarner engine ?  
Could be interesting for educationnal purpose but even if i have a look at the .md source, i don’t see how these file are structured. i guess we have to respect some nomenclature, some yarner keywords right ?

Cheers

---

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [March 26, 2021, 10:26am UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/7 "2021-03-26T10:26:30Z")

</div>

Thanks, @Nechoj and @nlb for your feedback!

## Literate Programming in general

@Nechoj Indeed, I forgot to mention the probably most widespread LP systems, which are Jupyter notebooks and RMarkdown. Both can run the code directly and allow to include the results (e.g. plots) into the document. However, both are more targeted on scripting, like for data analysis or other statistics.

With Yarner, I was more looking into the direction of writing software applications in compiled languages. Particularly simulation models in most of my personal use cases.

Of course, the approach taken makes things much easier as I do not need to care at all about the programming languages used, nor how to compile or run the code. This needs to be done by the user with the extracted code, just as usual.

## Yarner usage

@nlb There is a comprehensive and detailed [user guide](https://mlange-42.github.io/yarner/) that explains everything. But I can try to give a brief overview here.

### Installation

If you have Rust installed, simply run

```ruby
cargo install yarner

```

Otherwise, download it for your platform from the [releases](https://github.com/mlange-42/yarner/releases), extract it somewhere, and ideally put it on the `PATH`.

For more deteil, see chapter [Installation](https://mlange-42.github.io/yarner/installation.html) of the guide.

### Command line usage

Command line usage is extremely simple. To set up a new project, navigate into your to-be project folder and simply run

```ruby
yarner init

```

You then get a small but working example project which you can start with. To extract the code and create the “clean” documentation, simply run this, also inside the project’s folder:

```ruby
yarner

```

And that’s it… with the default configuration, you then find the extracted code in sub-folder `code`, and the clean docs in sub-folder `docs` (with the simple example project, “clean” docs look exactly the same as the Markdown sources).

As said, the code output can be compiled or run as usual. Further, the user is free to convert the Markdown docs to HTML or PDF, or simply push them to a Git forge where Markdown is rendered by default.

### Markdown syntax

Yarner tries to be completely compatible with standard Markdown. The basic functionality is based on named code blocks and macros.

Code blocks are named by their first line of code, prefixed by a user-configurable sequence. For the project presented here, I used `#-` as a prefix, as it renders as a comment with Ruby syntax highlighting. As an example, a code block named `Some block` would be written like this:

````markdown
```ruby
#- Some block
Your normal code in Some block...
```

````

Then, named code blocks can be used with macros (syntax is also configurable), which results in the macro being replaced by the content of the referenced block in the code output. As an example:

````markdown
```ruby
#- Main block
Some code here...
# ==> Some block.
```

````

The last line is a macro call to `Some block`. As a result, the content of `Some block` is put where the macro is. Thus, the extracted code would look like this:

```ruby
Some code here...
Your normal code in Some block...

```

> Note: Only fenced code blocks (surrounded with ````` or `~~~`) are supported, but not the indentation syntax (indent by four spaces, instead of fencing)

This allows you to structure the code for human readers, and Yarner takes care to re-structure it for the machine.

There are a lot more features, e.g. to allow to work with multiple source files, or to generate multiple code output files, but this is the fundamental concept how is works. For more details, see the [user guide](https://mlange-42.github.io/yarner/).

---

<div class="post-metadata">

**Author:** ![Aurolis](https://avatars.discourse-cdn.com/v4/letter/a/8797f3/32.png) [@Aurolis](https://in-thread.sonic-pi.net/u/Aurolis)\
**Post date:** [March 27, 2021, 6:26pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/8 "2021-03-27T18:26:57Z")

</div>

Cool ideas indeed! I must say though, that for me personally I think I need to really just code a track completely. I used trackers before (Amiga, PC etc) and I noticed that seeing the patterns restrict my creative process, a track evolves while listening to combinations of tones and effects for me.  
But that is just personal, I really like this project of yours for the sake of it being a very good idea!

---

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [March 27, 2021, 6:36pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/9 "2021-03-27T18:36:39Z")

</div>

Of course, I also started experimenting in Sonic Pi directly, and only after all the basics were set moved it to Literate Programming.

…if that is what you mean with:

> I think I need to really just code a track completely

Ah, ok… this is probably wrong and you mean the string patterns for drums and track structure…

EDIT: Actually, I just discovered that the uploaded track was not replaced by the latest version properly. For those who listened to it instead of pasting the code into SP, the best variation in the bass line (e.g. at 5:50) was missing until a few minutes ago…

---

<div class="post-metadata">

**Author:** ![Aurolis](https://avatars.discourse-cdn.com/v4/letter/a/8797f3/32.png) [@Aurolis](https://in-thread.sonic-pi.net/u/Aurolis)\
**Post date:** [March 27, 2021, 6:49pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/10 "2021-03-27T18:49:28Z")

</div>

I mean, if I see the structure of a song, my brain says: no, i dont want that, i will stop being creative now 😃  
That is why I was so amazed by Sonic PI, as that works for me! I think that is because I see music differently maybe and that’s why sonic pi suits me in that aspect

---

<div class="post-metadata">

**Author:** ![Buce](https://avatars.discourse-cdn.com/v4/letter/b/e5b9ba/32.png) [@Buce](https://in-thread.sonic-pi.net/u/Buce)\
**Post date:** [July 16, 2022, 3:17pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/11 "2022-07-16T15:17:48Z")

</div>

Hello and thank you for your very cool composition  
and good documentaion of this too.

I am trying to ‘yarner’ your seemingly incomplete ‘book’  
but this seems not to work just with the yarner 0.6  
(Windows program; .exe).

```ruby
C:\Users\ab2\Documents\Sonic Pi\Sonic-LP-main>yarner
[INFO] Extracting code from first-steps.md
[WARN] No entrypoint for file first-steps.md, skipping code output.
[INFO] Extracting code from first-steps/house-bounce.md
[INFO] Skipping unchanged file ../code/House/HouseBounce.rb
[INFO] Extracting code from introduction.md
[WARN] No entrypoint for file introduction.md, skipping code output.
[INFO] Extracting code from SUMMARY.md
[WARN] No entrypoint for file SUMMARY.md, skipping code output.
[INFO] Running plugin 'block-links'
[ERROR] Failed to run plugin command 'yarner-block-links': Das System kann die angegebene Datei nicht finden. (os error 2)

```

Probably I would need to install more of ‘yarner’?  
Thank you for some advise.

---

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [July 18, 2022, 11:45pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/12 "2022-07-18T23:45:05Z")

</div>

@Buce You need to install the plugin `yarner-block-links`. See [GitHub - mlange-42/yarner-block-links: Yarner plugin that creates links to all referenced end referencing blocks](https://github.com/mlange-42/yarner-block-links)

---

<div class="post-metadata">

**Author:** ![Buce](https://avatars.discourse-cdn.com/v4/letter/b/e5b9ba/32.png) [@Buce](https://in-thread.sonic-pi.net/u/Buce)\
**Post date:** [July 20, 2022, 5:06pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/13 "2022-07-20T17:06:45Z")

</div>

Thank you very much for the quick answer.

This works fine now.

How did you render/generate ([the book](https://mlange-42.github.io/Sonic-LP/))  
output then?

Somehow you say that github is rendering it automatically but could  
I extract this locally to have some html-files arround?

Thank for comments.

PS: You seem to have to unfinished chapters (dubstep-001 and morricone-001)…

---

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [July 22, 2022, 7:01pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/14 "2022-07-22T19:01:23Z")

</div>

You first run Yarner, then you run mdBook (runs on Yarner’s output).

Should be as simple as this:

```nohighlight
yarner
mdbook build

```

For more details, see the CI script: [Sonic-LP/publish.yml at main · mlange-42/Sonic-LP · GitHub](https://github.com/mlange-42/Sonic-LP/blob/main/.github/workflows/publish.yml)

You can also see the required plugins there for both, Yarner (`yarner-block-links`) and mdBook (`mdbook-toc`).

You can also download all these things manually that are retieved using `curl` there.

---

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [July 22, 2022, 7:03pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/15 "2022-07-22T19:03:03Z")

</div>

… and if you want to have it built and published by/on GitHub, just use the CI script. You need to enable GitHub Pages for it to work.

---

<div class="post-metadata">

**Author:** ![Buce](https://avatars.discourse-cdn.com/v4/letter/b/e5b9ba/32.png) [@Buce](https://in-thread.sonic-pi.net/u/Buce)\
**Post date:** [August 7, 2022, 6:54pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/16 "2022-08-07T18:54:11Z")

</div>

Unfortunately is mdbook-toc(v 0.9, 0.8) rejected by a windows virus detection? Any recommendations?

---

<div class="post-metadata">

**Author:** ![mlange](https://dub1.discourse-cdn.com/flex017/user_avatar/in-thread.sonic-pi.net/mlange/32/1899_2.png) [@mlange](https://in-thread.sonic-pi.net/u/mlange)\
**Post date:** [August 23, 2022, 10:25pm UTC](https://in-thread.sonic-pi.net/t/first-sonic-pi-track-made-with-literate-programming/5308/17 "2022-08-23T22:25:23Z")

</div>

No, never encontered that. Maybe add an exception?