First Sonic Pi track, made with Literate Programming

Thanks, @Nechoj and @nlb for your feedback!

Literate Programming in general

@Nechoj Indeed, I forgot to mention the probably most widespread LP systems, which are Jupyter notebooks and RMarkdown. Both can run the code directly and allow to include the results (e.g. plots) into the document. However, both are more targeted on scripting, like for data analysis or other statistics.

With Yarner, I was more looking into the direction of writing software applications in compiled languages. Particularly simulation models in most of my personal use cases.

Of course, the approach taken makes things much easier as I do not need to care at all about the programming languages used, nor how to compile or run the code. This needs to be done by the user with the extracted code, just as usual.

Yarner usage

@nlb There is a comprehensive and detailed user guide that explains everything. But I can try to give a brief overview here.

Installation

If you have Rust installed, simply run

cargo install yarner

Otherwise, download it for your platform from the releases, extract it somewhere, and ideally put it on the PATH.

For more deteil, see chapter Installation of the guide.

Command line usage

Command line usage is extremely simple. To set up a new project, navigate into your to-be project folder and simply run

yarner init

You then get a small but working example project which you can start with. To extract the code and create the “clean” documentation, simply run this, also inside the project’s folder:

yarner

And that’s it… with the default configuration, you then find the extracted code in sub-folder code, and the clean docs in sub-folder docs (with the simple example project, “clean” docs look exactly the same as the Markdown sources).

As said, the code output can be compiled or run as usual. Further, the user is free to convert the Markdown docs to HTML or PDF, or simply push them to a Git forge where Markdown is rendered by default.

Markdown syntax

Yarner tries to be completely compatible with standard Markdown. The basic functionality is based on named code blocks and macros.

Code blocks are named by their first line of code, prefixed by a user-configurable sequence. For the project presented here, I used #- as a prefix, as it renders as a comment with Ruby syntax highlighting. As an example, a code block named Some block would be written like this:

```ruby
#- Some block
Your normal code in Some block...
```

Then, named code blocks can be used with macros (syntax is also configurable), which results in the macro being replaced by the content of the referenced block in the code output. As an example:

```ruby
#- Main block
Some code here...
# ==> Some block.
```

The last line is a macro call to Some block. As a result, the content of Some block is put where the macro is. Thus, the extracted code would look like this:

Some code here...
Your normal code in Some block...

Note: Only fenced code blocks (surrounded with ``` or ~~~) are supported, but not the indentation syntax (indent by four spaces, instead of fencing)

This allows you to structure the code for human readers, and Yarner takes care to re-structure it for the machine.

There are a lot more features, e.g. to allow to work with multiple source files, or to generate multiple code output files, but this is the fundamental concept how is works. For more details, see the user guide.

2 Likes