Intro: Getting started w/Sonic-pi

I just found Sonic-Pi yesterday. I had some trouble getting the portable version working–I’ll investigate that later–but I managed to get the traditional Windows installer version working fine.

I am a blind user, so I was particularly interested when Sam said he was interested in implementing better screen reader support. I therefore signed up on Patrion as a beta tester to see if I could help. I’ve used ChucK & believe me, screen reader support in Sonic-Pi is a whole lot better, though it could still use some work. Primary difficulties are navigating between windows using the keyboard & the fact that blank lines don’t announce themselves, which can be a pain-&-a-half when trying to enter code above or below other code. That having been said, it is thoroughly usable so long as you bear in mind its quirks when being used w/a screen reader, & for that I am deeply appreciative. Thank you.

I’m sure I’ll become even more proficient as I continue to experiment–a day isn’t very long in this arena–& already it’s been a pretty fun experience. Although many blind folks are musicians, music technology has lagged behind for us in the extreme in terms of accessibility. So being able to use Sonic-Pi feels like a real treat instead of yet another door being slammed in my face. I’ll likely be hopping on here w/queries over time, but for now, I find the tutorials easy to understand & follow. Good work, & thanks again.

5 Likes

Hi @abletec,

Welcome to our community, it’s lovely to have you here. I really do hope you stick around :slight_smile:

It’s fab to hear that your able to get things working via your screen reader. Accessibility is really important to me, so please know that I read your message with a huge warm and happy smile.

I also recognise that there’s much more work to be done. I would be really grateful if you were able to help in any way so that we can continue to make Sonic Pi more accessible.

For example, you say that blank lines are not discoverable. How are they normally observed and communicated by screen readers? Is there a simple trick we could do - such as insert a tab on each line to make things easier whilst we work on a more appropriate solution?

I personally consider every aspect of the system that to can’t interact with (or don’t find the interaction simple and intuitive) to be a bug and will treat it as such. It would therefore be amazing if you could let me know what doesn’t work and what could be improved. Especially if there are obvious things we could change that would make your experience richer and ultimately more fun!

I’m also really looking forward to hearing what you make with Sonic Pi!

Happy live coding!

Hi, Sam. 1st, let me, if I might, address an issue w/the forums for a moment. I tried really hard to start this topic in the category of introductions & stories. You saw how successful that was. :rolleyes: So I do apologize for where it is, I promise I knew better & I tried mightily to put it where it actually belonged. So yeah–the accessibility of the categories part isn’t quite there yet lol.

If I were to prioritize problems in terms of screen reader access w/Sonic-Pi, the inability to set focus to the various windows via the keyboard is surely at the top of the list. As an experiment, open it up & try to use something like ctrl tab or f6 or ctrl f6 or even menu bar options to navigate to & then set focus in the various windows. It won’t happen. You can obviously toggle the visibility of the various windows using the menu, but you’re not actually setting the focus when you do that, if I make myself clear. The only way one can traverse the various windows is via the mouse. For screen readers, it’s clunky at best & requires knowledge that many screen reader users simply don’t have. & it surely won’t work in a live coding situation. The rest of the stuff I’ve found, at least thus far, is just pita type stuff. This 1 borders on being a show stopper & will be that indeed in a live event. So I think from an accessibility standpoint at any rate, that one’s numero uno. Obviously the best solution is to have some hot keys that will go to the various windows in question, ie, alt e for code editor, alt p for preferences–or similar–the next best would be using something like ctrl tab & ctrl shift tab to navigate back & forth between the windows, but in a live event, it’s actually likely best to just have a quick keypress.

I also don’t think I’m seeing error messages, & I’ll be investigating that further expeditiously & get back to you w/my findings regarding that aspect, because that’s obviously pretty immportant as well. The various aspects of code/syntax highlighting obviously don’t exist for us at this point.

In terms of the line breaks, I rather suspect we ourselves might be able to fix this by putting entries in our default speech dictionaries. I’ll be experimenting at some point to see if that solution indeed works.

I would be willing to write an article for you to post pertaining to use of Sonic-Pi w/a screen reader, if that’s of interest, though I do think I likely need a bit more time to experiment, figure out where the major hurdles are, as well as workarounds, if any.

Lastly, if you need hosting for sonic-pi.net, hit me up off list & we’ll see what we can’t scare up for you.

1 Like

Super useful feedback, thanks. wrt to the forums, this isn’t something I develop. It’s a forum called Discourse and their community chats here: https://meta.discourse.org I’m sure they would love to hear your feedback, and any improvements they can make will be available to all communities that use their forum software.

With respect to Sonic Pi let’s tackle the focus thing first.

I honestly don’t know how to tell if something is in focus or not. Even with a mouse, it’s not clear what’s currently in focus and what’s not and I wouldn’t know how to test whether a shortcut I set up would even work.

The only aspect of focus I can clearly see is the presence of a cursor in the editor window. Other than that, the notion of focus is something that’s completely invisible and hasn’t been a concept I’ve needed to be aware of until now.

Oh, and an article about using Sonic Pi with a screen reader would be amazing and something I’d love to see - especially if it both helped other screen reader users get over the initial bumps that we haven’t fixed yet and also lets us know what needs to be fixed!

Sam, this is a concept you know well–you just don’t know it by the term I’ve used. When an item on a menu bar is highlighted, that is the focus object. Your cursor in an edit window is a focus object. Basically, any object you’re interacting with is the focus object. That object is usually in a window, so the window containing the focus object is the focus window. The problem comes that if, for example, you get an error message, you’re alerted to that by a blinking red color, perhaps. You simply look at the text & read it, fix the mistake or whatever, & go on about your business. Now–as screen reader users, we clearly cannot train our eyes on the spot in question. So therefore our screen reader either needs to read it aloud or it needs to be able to go to the window containing the text so we can use our arrow keys, etc, in order to read/interact w/the information being presented. Preferably both,ie, you’re alerted to a problem but you can also go to the window in order to be able to study it in greater depth, just as you would when looking at the information on your screen. So I have to set that window as the focus window for the screen reader so it can read/interact w/the text being presented. If you think about the screen reader having to do the looking for us, it might provide some help w/regard to thinking about the issue in question.

The help window may be another place where keyboard focus is needed. You could just click a sample or synth name w/the mouse. In order for me to choose 1, however, I’d need to be able to use my arrow keys, the highlighted option should speak, & I should then be able to press the enter key in order to make my choices. I’ll need to check & see if it reacts thus, & I will do so as work permits.

So what I discovered when intentionally introducing an error is that obviously the code won’t run. The silence is my error alert. I was able to use the menu bar to go to the info window where it said I had an error on line 13. Well, no, not really, as I had intentionally left out a ‘do’ on line 5, but compilers/interpreters are pretty notorious for getting this stuff wrong sometimes, so no biggie.

Another example is getting to the code edit window. When Sonic-Pi first opens, there are a series of windows on screen–log, cues, etc. It doesn’t immediately go to the code edit window–it seems I’ve got to do the equivalent of a mouse click to get there. It’d be nice to be able to press a hotkey & get to the window in question. How to do that is a little bit code-dependent.

Am I correct you’ve written Sonic-Pi in Ruby? It’s not a language w/which I’m very familiar, but, if required, I’ll try to get myself up to speed enough so that I can at least understand how it deals w/interface objects, etc in order to try to lend a bit more assistance here.

I was aware the forums weren’t developed by you. That was just more of an apology for the topic being in the wrong forum, & why that was the case. I mean–you know–I’m more than capable of doing stupid stuff–but in this case, it wasn’t actually me for a change. Lol. It would be good if they’d make those accessible, though. That’s for another day, however.

Thanks for the explanation - what you describe is exactly how I imagined it might be - the system must have some state representing where the user is so that it can tell the screen reader what the various options are. Kind of like moving in and out of different rooms in a MUD. What you’re asking for is either nice organised series of doors connecting the rooms, or ideally an instant teleport device.

With respect to debugging this, one thing that wasn’t clear to me was knowing which panel or window was currently in focus. With standard OS windows, this is clear as the OS clearly communicates this to you visually. However, within Sonic Pi’s GUI window, the various panes do not communicate anything visual with respect to focus. For the purposes of debugging I’ve just modified my local stylesheet to place a red border around the currently focussed panel and now I can see what’s going on. Clicking with my mouse allows me to change the focus and this is now visible to me. One thing of note, is that I can’t currently seem to put the preferences panel in focus. Are you able to access this in any way? Perhaps the focus border styling isn’t working properly here?

Anyway, I believe I have enough to go on to at least improve things. With respect to shortcuts, how does this sound:

  • Control-Tab - move focus to next widget
  • Control-shift-Tab - move focus to previous widget
  • Control-shift-e - move focus to editor
  • Control-shift-l - move focus to logs
  • Control-shift-c - move focus to cues
  • Control-shift-p - move focus to preferences
  • Control-shift-h - move focus to help listing (e.g. the list of available help section)
  • Control-shift-d - move focus to help details (e.g. the contents of the chosen help section)
  • Control-shift-w - move focus to syntax/runtime error warnings

Would it also make sense to add these shortcuts to the main app’s menu? If so, under which heading would be most natural for you? Window, App, Navigation, something else?

Also, with respect to the actual shortcuts, I’m very happy for you to suggest alternatives. One thing to consider is that we already have a pretty extensive list of shortcuts which I don’t want to clash against: http://sonic-pi.net/tutorial.html#section-B-2

One final question - with editor, preferences or the help are you able to access the tab selection buttons? In the editor there are the various editor buffers: |0|, |1|, |2|, |3|, etc. in the preferences there are: Audio, IO, Editor, Visuals, Updates. In the help there is: Tutorial, Examples, Synths, FX, Samples, Lang. My “highlight focus” style trick doesn’t seem to highlight these menu widgets, so I’m unsure if these can be focussed or not.

Oh, and to answer your query about the implementation language. It’s true that the coding language is based on Ruby. However, the GUI is built with Qt in C++. Qt has extensive accessibility functionality so I’m absolutely positive we can massage things into a state that isn’t just acceptable but is enjoyable to use and perform with.

Hi @abletec

as i don’t know names of screen reader, could you tell us which screen reader are you using ? What software ?
i met a blind old man 5 years ago and just to visit a website, i was impressed by the patience he had to listen to every words announcing each item of each menu. before be able to reach the main content.

Cheers

Hi, all. Sam, I’m kind of impressed! Qt has really come a very long way. It used to be that whenever we blindies heard the word “Qt”, it was an acccessibility death sentence–we knew that even attempting to use the program would be utterly useless. That was Qt 4. It was truly an accessibility nightmare-&-a-half. Qt 5 has obviously progressed a very long way.

As you know, objects have properties &/or states. 1 such state is “is focusable”, or can the particular object in question gain focus. That is the first prerequisite toward making the window/widget focusable.

Basically, what happens is that objects have to expose themselves to the screen reader. Yes, I am aware of the family-friendly nature of this forum, & no–I’m not being obscene lol. It seems as though Qt has done a good job of implementing accessibility into the interface, so as not to require too much work on the programmer’s part in order to make all that happen, & that’s a good thing. In the event you’ve got a bad case of insomnia, here’s some reading that should cure that.
https://doc.qt.io/qt-5/accessible.html
& there are other links on that page as well if that particular page didn’t serve sufficiently as a sleep aid.

The screen reader sees the tabs you reference, but clicking in them doesn’t seem to do much. Can you please give me a scenario as to how these are useful &/or how I can test my ability to access the contents?

I can access the preferences window, but, again, it all requires clicking. The checkboxes contained therein are not focusable in any way using the keyboard, say, for instance, the tab/shift-tab keys. So that’s clearly less than optimal.

@nlb, never do I listen to every single word on a webpage. That indicates to me a profound lack of knowledge on how to make best use of the screen reader. Unfortunately, many blind folks–indeed, the majority–do not receive adequate training in this regard.

1 free screen reader available for windows is NVDA (Nonvisual Desktop Access) available from
nvda-project.org. Narrator is built into Windows & is improving by leaps & bounds, though I personally don’t find it useful in many instances. Orca is available on Linux, & Voiceover is built into the Mac & IOS operating systems.

In the preferences the first two tabs are:

  1. The Audio tab reveals:
    • Master Volume slider
    • Synths & FX checkboxes (safe mode, enforce timing guarantees, enable external synths/FX)
    • Audio Output checkboxes (invert stereo, force mono)
  2. The IO tab reveals
    • MIDI Ports (a list of MIDI in and out ports, reset MIDI button)
    • Midi Configuration (enable midi subsystems checkbox default MIDI channel selection box)
    • Networked OSC (enable OSC server checkbox, Send/Receive remote OSC checkbox, OSC IP address and port info)

In the help widget, first two tabs are Tutorial and Examples. Tutorial lists all the tutorial sections and Examples lists all the available code examples. In the editor, the different tabs switch between different editor windows - there are 10 of them to work with.

Is any of this accessible to the screen reader?

Hi there,

another advantage of Sonic-Pi in general. Using code as musical expression makes it easier for blind people to produce music, as it’s just text. I hope, you keep up the feedback @abletec. Would be interesting to know how well it works in the end together with Sam’s advice and maybe changes to Sonic-Pi.
I wish you a successfull start in Sonic-Pi.

Greetings.

Hi @abletec,

I’ve uploaded a fresh new beta to Patreon which has some initial support for changing focus via shortcuts and the app menu.

The new shortcuts are:

  • Control-shift-e - move focus to editor
  • Control-shift-l - move focus to logs
  • Control-shift-c - move focus to cues
  • Control-shift-p - move focus to preferences
  • Control-shift-h - move focus to help listing (e.g. the list of available help section)
  • Control-shift-d - move focus to help details (e.g. the contents of the chosen help section)
  • Control-shift-w - move focus to syntax/runtime error warnings

I’d love to know what you think - what works and what doesn’t.

Also, if you have time, I’d still love to know what (if anything) you can access wrt to the preferences and help widget (see post here: Intro: Getting started w/Sonic-pi)

As mentioned before, I’d like to see Sonic Pi’s accessibility support improve with every release, and this beta is making good on that promise :slight_smile:

2 Likes

Sam, thanks. I’ll paw through my emails & see if I can find a link to download betas. Once I figure all that out, & get it downloaded & installed up, I’ll provide feedback. Again, thanks.

@abletec - here’s the link: https://www.patreon.com/posts/sonic-pi-v3-2-1-34976693

Ok, well finally I got a chance to download the beta. Sam, awesome. I’m definitively able to set focus to the various windows now.

So as to the preferences & help windows, I can get to them now. Accessing the controls in the window is a bit more problematic. Basically, I’ve gotta pretend I’m using a mouse, & it’s hit or mouse (& usually miss) lol. The good news about that though is that screen reader can read the text in the window, so we can definitely make this fully accessible. But the shortcuts work as expected, & that’s huge progress!

Thanks so much. This is great!

I am currently changing the platform for the blind musicians’ list I own from my server to freelists. I’m gonna wait till everyone gets subscribed up & things settle down a bit, & then I’ll notify them about Sonic-Pi & how you’re making it accessible to us. I would sincerely hope some Patrien donations will come your way as a result. I’ll be pretty angry if not.

You’ve got tabs labeled 0-9 in the editor. What are those for (buffers, I think?), but how do you access them & what do you do once you have? Seems like whenever I pretend to use the mouse & click, nothing happens.

1 Like

Hi Abe,

The buffers (0-9) are area’s where you can write and run code.

They are very flexible, for example:

You could load a different song into buffers 0-9 and use an outside
control such as a python script to make a jukebox.

Or you could be writing the song in buffer 0, and testing different
parts in the other buffers before cut and pasting them into buffer 0.

Moving beyond these simple examples, the buffers can share variables,
rings etc, to a certain extent, so if buffer 0 runs out of line space for new code,
you can overflow into another buffer in a limited way by following certain rules.

To use them, select a buffer by clicking on its number, then start writing your code
in the editor window above, or select the load button to bring back a previously saved
piece of code into the selected buffer. Click the run button to play or test your code.

Save should be obvious, however Record allows you to run your song and save it as a
.wav file once completed. Makes for easy upload to Soundcloud, and other places.

Hope this has been of help.

Regards,

Eli…

@abletec - this is wonderful to hear! Thanks for taking the time to download the latest beta and for sharing your feedback.

So what do you think the next actionable point for me is?

Sounds like there’s still work to be done in the following areas:

  1. Preferences
  2. Help
  3. Editor buffers

For each of these, it seems like falling back on Qt’s default accessibility behaviour isn’t quite enough. Or, there’s something inadvertantly funky I’m doing to override the default behaviour and reduce accessibility.

What would your highest priority be for my next action point? Also, what would be a good way of testing I’ve managed to address this action point?

I’m so happy we’re making progress!

1 Like

Sam, as I said, QT doesn’t exactly have a stellar reputation for accessibility.

1 of the ways you can test the software is to simply unplug clicky Micky. Can you then access the various parts using just your keyboard? If not, then you know a blind person, or anyone for that matter who can’t use a mouse, will have trouble. So for example in the ‘preferences’ window, you have the audio output tab, where you you have controls like ‘Invert Stereo’ & ‘Force Mono’. Can you access these w/o benefit of Mickey? If not, (& I’d actually wager you can’t), (& you have no idea how bad I hate losing money & thus how certain I am of my statements :)), then those would be your next benchmarks.

1 thing that I can tell you right now that would be extraordinarily helpful is to have the focus window name appear in the title bar, so we’re always certain of where we are.

But this is a truly fantastic start!

I’m trying to teach SonicPi to a 9 year old starting with twinkle twinkle. For the experienced teachers, do you have a curriculum you follow ? Any current educational setups we can zoom once a week in to?

Have a look at https://sonic-pi.mehackit.org/

1 Like