News for hackers. Stuff that matters.

A comment on WhisperSpeech – An open source text-to-speech system built by inverting Whisper.

Re:WhisperSpeech – An open source text-to-speech system buil... by lolinder on Wednesday January 17, 2024 @11:32PM (#39037879)
I know it's old at this point and doesn't use the fancy new tech, but Mycroft's Mimic 3 is still pretty impressive and is small enough to fit comfortably and generate speech in real time on a raspberry pi [0]. Some of their voices are better than others, but the best of them are definitely equal to the examples of WhisperSpeech given here.

[0] https://mycroft.ai/mimic-3/

[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by follower on Thursday January 18, 2024 @01:26AM (#39038614)
If you're not already aware, the primary developer of Mimic 3 (and its non-Mimic predecessor Larynx) continued TTS-related development with Larynx and the renamed project Piper: https://github.com/rhasspy/piper

Last year Piper development was supported by Nabu Casa for their "Year of Voice" project for Home Assistant and it sounds like Mike Hansen is going to continue on it with their support this year.

[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by IshKebab on Thursday January 18, 2024 @08:04AM (#39041185)
That is much better quality than Mimic (which didn't sound very close to WhisperSpeech to me).
[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by lolinder on Thursday January 18, 2024 @10:28AM (#39042907)
Wow, I did not know that! Thank you!
[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by boxed on Thursday January 18, 2024 @02:13AM (#39038906)
The "English US" voice sounds more Scottish than American to me :P
[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by rcarmo on Thursday January 18, 2024 @02:55AM (#39039167)
Which might be a good thing, nae, laddie?
[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by jpcl on Thursday January 18, 2024 @06:23AM (#39040490)
Yeah, the Mimic is a lot less resource intensive. We are working to improve WhisperSpeech in this regard but it's probably always going to require more compute (but in return you'll get higher quality).

That said if you have a modern NVidia GPU you should be able to run a voice-bot in real-time with WhisperSpeech.

[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by freedomben on Thursday January 18, 2024 @10:59AM (#39043354)
Will something like whisper.cpp be possible for whisper speech?
[ Parent | on HN ]
Re:WhisperSpeech – An open source text-to-speech system buil... by jpcl on Thursday January 18, 2024 @12:39PM (#39044790)
We looked at this at one point and it seems whisper.cpp/llama.cpp have all the bits needed to make it work. I'd love to help if someone wanted to give it a shot.
[ Parent | on HN ]