Just have llama.cpp emit an “um”, “uhh” etc. when the buffer’s down to one word. :D

you laught at the end but I love this solution

Humans have loved the same solution since we first started talking, as well

Don’t forget to mix it with “apparently”, “you know what I’m saying”, “I mean”, “you know” etc.

I can't tell if you're disparaging the usage or not (truly, I can't tell), but such utterances exist because they serve a real function. Disfluency is an integral part of speech.

I think it's a good idea, if done well. It could also potentially be combined with dynamically adjusting speed of the speech, and reducing or increasing the use of shortcuts and contractions, making word replacements.

I know wish for a model built to be a low-computation filter that takes text in and produces padded text out intended for TTS and annotated with pauses or sounds and extra words that maintains the same meaning but provides the ability to dynamically adjust the level of verbosity to maintain a fixed rate of words per minute.

I always thought of them as the human equivalent of hard drive noises. <brrrrr brrbrrbr>

array_rand($verbal_fry[$locale]) /* :D */

Timing and emphasis work better if you know where the sentence is going; otherwise you sound like one of those translators at the UN with the flat stream of words.

I suppose it could be buffered, and used to only shorten the wait in cases with long responses, rather than aiming for perfect and push out words as they come. Besides, GPTs sound like translators anyway.

I mean.. 99% of the current TTS engines wouldn't know timing and emphasis if it hit them.

Besides humans do this all the time, we start saying words before we even have, uhh, any idea how we're gonna end the sentence and for the most part it, uhh, works out. Should be doable.

Also there's a big difference between waiting for a buffer of outputs and waiting for all the outputs.

The difference is that if you are the one saying the sentence, you at least have some idea where it's going. Receiving one word at a time from a different source isn't the same.

> you at least have some idea where it's going

Unless you're Michael Scott.

ElevenLabs and Gemelo.AI are services that both support text input streaming for exactly this use-case. I am not aware of any open-source Incremental TTS (this is the term used in research afaik) model but you can already achieve somthing similar by buffering tokens and sending them to the TTS model on punctuation characters.

ElevenLabs only has streaming output available. I've had a look at both recently and ElevenLabs doesn't have streaming input listed as a feature. Would be cool if it had it, though. You could probably approximate this on a sentence level, but you would need to do some normalisation to make the speech sound even.