MaryOSC - Text to Speech OSC server

I was trying to find text synthesizer that supports OSC but didn’t find any … so I made one. It’s called MaryOSC and it uses MaryTTS to do the hard parts. It got some nice effects and you can also save the created audio to files on the fly so you can use Sonic Pi effects also in that way.

Here are some examples:

Loop all voices:

use_osc "localhost", 9000

osc "/mary/effects", "Whisper(amount:50)+Robot(amount:100)"

live_loop :all_voices do
  osc "/mary/voice", tick
  osc "/mary/say", "binary dreams"
  sleep 1
end

use_osc "localhost", 9000

osc "/mary/effects", "Volume(amount:0.5)+Stadium(amount:100)"

arr1 = ["dog","cat","cow","pig","fox","chicken","bunny","worm"]
arr = ["sheep","duck","bunny","chicken","wolf","deer","snail"]

live_loop :random do
  osc "/mary/voice", "cmu-slt-hsmm"
  osc "/mary/say", arr.choose+arr1.choose
  sleep 2
end

live_loop :freaky do
  tick
  sample :bd_zome if (spread [4,8].choose,16).look
  sample :bass_dnb_f if (spread [2,3].choose,16).look
  play [40,42,46].choose if (spread [5,6].choose,16).look
  sleep 0.25
end

Creating audio files on the fly:

use_osc "localhost", 9000

osc "/mary/save", "~/samples/mary/goodnight.wav", "Good night!"
sleep 1
sample "~/samples/mary/goodnight.wav"

I hope you all like it. Good night :wink:

This looks intriguing, and I would love to give it a go. Could you give a bit more hand-holding on setting up MaryTTS and MaryOSC from scratch. I’m not really into Java.gradlew)

EDIT 2
I managed to get it going (once I’d set the executable bit on gradlew)

Nice system!

Thanks for testing. I marked the gradlew as executable to the git in hope that it helps other linux/mac users.

Found out that the created sample can also be reused by using sample_free. Makes it possible to change the content:

use_osc "localhost", 9000

s = "~/samples/mary/rambles.wav"
t = ["believe","dream","forget","dance","die","sleep","eat","live","code"]

live_loop :samples do
  osc "/mary/save", s, t.choose
  sleep 1
  sample_free "~/samples/mary/", :rambles
  with_fx :echo, decay: 3 do
    sample "~/samples/mary/", :rambles, pitch: [0.5,1.0,1.5,2.0,3.0,4.0].choose
  end
end

I quess you could also use sample_loaded in some cases to wait until the sample is available.

The other thing I’d like to work out is how to loopback the audio output of mary to Sonic Pi. I have a loopback utility on my Mac but it can’t see the audio output of the mary,tts system.

I’ve just completed The Twelve Days of Christmas with the verses generated musically by Sonic Pi and the text spoen by mary,tts. I would like to feed the speech back into S for further processing there, and to make it easier to balance the relative audio levels.

I can see “Java Platform Binary” Application in Win10 audio settings and change the volume from there. There’s also option to redirect voice to other ports … but VB virtual audio cable doesnt seem to work or i just dont know how to use it.

There is also the inbuilt “Volume” effect that can be used with OSC.

Application is using marytts AudioPlayer to produce the sound. Have to look what other options there might be.

Here is my first effort using your excellent Text to Speech OSC server.

Sounds good and in sync! Bit monotonic though … but changing the pitch with effects etc. is a lot of work. Im planning to study marytts more and come up with some easier way to make the spoken voices more melodic. Something like this would be nice:

osc "/mary/sing", "do re mi fa so la ti do", [1,2,3,4,5,6,7,8], :c, :major

Meanwhile there is the F0Scale and F0Add effects that can be used to manipulate the pitch. BTW, there is also online demo that can be used to try out different effects (click “Show audio effects” from the bottom of the page). MaryOSC is using the same hidden markov model voices (hmm) which can be found under the voice menu. There are also other sampled voices which are not included because those cannot be loaded automatically trough maven repositories (but could be downloaded from the marytts homepage and included as jars in gradle.properties if someone wants to try those out).

Hi :slight_smile:

anyone has tried it on Windows? I’m having troubles running the server :frowning:

I opened an issue on the github repository https://github.com/amiika/MaryOSC/issues/1 for more details.

Any idea of what could be the issue?

Hi notoriusvcnt! Welcome to the community!

Im running Sonic Pi and MaryOSC on Windows 10, so it should be working. This issue is most likely related to the missing dependencies.

I fixed some minor issues and updated the code. Now it should at least fail faster if the dependencies are missing. Try these to fix the problem:

git pull
gradlew build --refresh-dependencies
gradlew run --console=plain

Got Sonic Pi triggering this nicely on Windows 10 :+1:
I’m definitely keen on being able to route the output to Sonic Pi so that I can layer Sonic Pi FX onto it - I haven’t figured out how or if it’s possible with VB audio virtual cable either…

Great! Note to others who might want to try: MaryOsc requires Java 1.8 JDK to run (OpenJDK 12 might also work).

Buhh … tried hours of configuring those virtual audio channels on Win10 and gave up. Also been having problems with TidalCycles which basically requires multiple channels to work properly.

However, saving the synthetized voice as samples using /mary/save is a great workaround. You can also create array of texts etc. and save those and run those trough FX.

1 Like