News for hackers. Stuff that matters.
Re:OpenAI Audio Models by kibbi on Thursday March 20, 2025 @04:14PM (#43428353)
Large text-to-speech and speech-to-text models have been greatly improving recently.

But I wish there were an offline, multilingual text-to-speech solution with good voices for a standard PC — one that doesn't require a GPU, tons of RAM, or max out the CPU.

In my research, I didn't find anything that fits the bill. People often mention Tortoise TTS, but I think it garbles words too often. The only plug-in solution for desktop apps I know of is the commercial and rather pricey Acapela SDK.

I hope someone can shrink those new neural network–based models to run efficiently on a typical computer. Ideally, it should run at under 50% CPU load on an average laptop that’s several years old.

The same goes for speech-to-text. Whisper.cpp is fine, but last time I looked, it wasn't able to transcribe audio at real-time speed on a standard laptop.

I'd pay for something like this as long as it's less expensive than Acapela.

Re:OpenAI Audio Models by ZeroTalent on Thursday March 20, 2025 @04:22PM (#43428438)
Look into https://superwhisper.com and their local models. Pretty decent.
Re:OpenAI Audio Models by kibbi on Thursday March 20, 2025 @04:33PM (#43428595)
Thank you, but they say "Offline models only run really well on Apple Silicon macs."
Re:OpenAI Audio Models by ZeroTalent on Thursday March 20, 2025 @06:16PM (#43429636)
Many SOTA apps are, unfortunately, only for Apple M Macs.
Re:OpenAI Audio Models by 5kg on Thursday March 20, 2025 @04:23PM (#43428448)
May I introduce to you

https://huggingface.co/canopylabs/orpheus-3b-0.1-ft

(no affiliation)

it's English only afaics.

Re:OpenAI Audio Models by kibbi on Thursday March 20, 2025 @04:39PM (#43428669)
The sample sounds impressive, but based on their claim -- 'Streaming inference is faster than playback even on an A100 40GB for the 3 billion parameter model' -- I don't think this could run on a standard laptop.
Re:OpenAI Audio Models by dharmab on Thursday March 20, 2025 @05:22PM (#43429136)
I use Piper for one of my apps. It runs on CPU and doesn't require a GPU. It will run well on a raspberry pi. I found a couple of permissively licensed voices that could handle technical terms without garbling them.

However, it is unmaintained and the Apple Silicon build is broken.

My app also uses whisper.cpp. It runs in real time on Apple Sillicon or on modern fast CPUs like AMD's gaming CPUs.

Re:OpenAI Audio Models by kibbi on Thursday March 20, 2025 @06:13PM (#43429608)
I had already suspected that I hadn't found all the possibilities regarding Tortoise TTS, Coqui, Piper, etc. It is sometimes difficult to determine how good a TTS framework really is.

Do you possibly have links to the voices you found?

Re:OpenAI Audio Models by dharmab on Thursday March 20, 2025 @09:08PM (#43430862)
Re:OpenAI Audio Models by wingworks on Thursday March 20, 2025 @07:18PM (#43430149)
Did you try Kokoro? You can self host that. https://huggingface.co/spaces/hexgrad/Kokoro-TTS
Re:OpenAI Audio Models by kibbi on Thursday March 20, 2025 @08:36PM (#43430672)
Thanks! But I get the impression that with Kokoro, a strong CPU still requires about two seconds to generate one sentence, which is too much of a delay for a TTS voice in an AAC app.

I'd rather accept a little compromise regarding the voice and intonation quality, as long as the TTS system doesn't frequently garble words. The AAC app is used on tablet PCs running from battery, so the lower the CPU usage and energy draw, the better.

Re:OpenAI Audio Models by SamPatt on Friday March 21, 2025 @09:00AM (#43435128)
Definitely give it a try yourself. It's very small and shouldn't be hard to test.
Re:OpenAI Audio Models by Ey7NFZ3P0nzAe on Friday March 21, 2025 @09:08AM (#43435221)
I heard good things about fish audio