News for hackers. Stuff that matters.
Re:A CC-By Open-Source TTS Model with Voice Cloning by asaddhamani on Saturday November 09, 2024 @01:25AM (#42092851)
From a quick try results aren't good. Sounds bland, and the text I type isn't exactly equal to the text that is spoken. Didn't try with voice cloning though.

Why is good TTS so expensive and why are there no good open source options? Is it just from the need for high quality training data? I don't imagine these models are more expensive to run compared to SOTA LLMs, yet they cost so much more.

Re:A CC-By Open-Source TTS Model with Voice Cloning by modeless on Saturday November 09, 2024 @01:42AM (#42092895)
There are a lot of options. StyleTTS2 is pretty good, XTTSv2 is pretty good, the new E2 TTS and F5 TTS also seem decent.
Re:A CC-By Open-Source TTS Model with Voice Cloning by amrrs on Saturday November 09, 2024 @02:24AM (#42093023)
Commercially available high quality training dataset is the key. Open search libraries don't get the luxury of working with voice actors to record voices.
Re:A CC-By Open-Source TTS Model with Voice Cloning by Aeolun on Saturday November 09, 2024 @02:35AM (#42093069)
Would it be hard to create such a training dataset? Seems like you’d just need a lot of people to say a bunch of stuff for you?
Re:A CC-By Open-Source TTS Model with Voice Cloning by wahnfrieden on Saturday November 09, 2024 @02:42AM (#42093098)
needs a crowdsourced model
Re:A CC-By Open-Source TTS Model with Voice Cloning by huggingmouth on Saturday November 09, 2024 @03:26AM (#42093254)
Ideally, Mozilla would step up here given their mission statement, but they won't, probably because their CEO needs another bonus.
Re:A CC-By Open-Source TTS Model with Voice Cloning by IshKebab on Saturday November 09, 2024 @03:56AM (#42093346)
Yeah there's no chance Mozilla would do anything like this:

https://commonvoice.mozilla.org/

Re:A CC-By Open-Source TTS Model with Voice Cloning by mgkimsal on Saturday November 09, 2024 @11:28AM (#42095249)
That's the first thing I thought of! I wonder how used these are. Are there any sources or data points indicating that this commonvoice data is being used, and if so, where/how? I think I may have contributed to this a few times back years ago. Nice to see it's still going, would be better to know it's being used.
Re:A CC-By Open-Source TTS Model with Voice Cloning by willwade on Sunday November 10, 2024 @03:26AM (#42099084)
It was used quite a bit of speech to text - but tts it’s not that great.
Re:A CC-By Open-Source TTS Model with Voice Cloning by Aeolun on Sunday November 10, 2024 @05:46AM (#42099519)
It costs a million dollar a year to host 32k hours of audio?
Re:A CC-By Open-Source TTS Model with Voice Cloning by em-bee on Saturday November 09, 2024 @04:26AM (#42093434)
a few weeks ago i used piper to create an acceptable translation of a book. i didn't listen to it all, but the result sounded better than anything i was able to listen to before. good enough to listen to a book if a human read one is not available. just a few years ago, this was not the case.

in other words, while FOSS TTS lags behind commercial options, it does get better and i expect within a few years it will produce results that are at least as good as the commercial options today if not fully caught up.

Re:A CC-By Open-Source TTS Model with Voice Cloning by asaddhamani on Saturday November 09, 2024 @06:35AM (#42093858)
Piper seems roughly equivalent to old-school TTS outputs that sound flat, jumpy with the concatenative approach. Listen to this first example I tried:

https://rhasspy.github.io/piper-samples/samples/en/en_GB/ala...

Of all the TTS APIs I have tried, I like OpenAI voices the best. Haven't considered things like elevenlabs because I find them ridiculously expensive.

I love voice to voice interfaces, but only when they sound natural to my ears, and the current pricing for good ones is prohibitive for a huge number of use cases.

Re:A CC-By Open-Source TTS Model with Voice Cloning by em-bee on Saturday November 09, 2024 @07:53AM (#42094162)
well, i was comparing it to the free tools available a few years ago, and against that, this example is a markable improvement. it's the first that i could actually bear to listen to over a longer period of time. i expect just another few years and this will actually be good.
Re:A CC-By Open-Source TTS Model with Voice Cloning by sjnair96 on Saturday November 09, 2024 @05:12AM (#42093576)
Have you tried VoiceCraft?
Re:A CC-By Open-Source TTS Model with Voice Cloning by asaddhamani on Saturday November 09, 2024 @06:28AM (#42093828)
Yeah all these seem hyper focused on "voice cloning" so on replicate VoiceCraft doesn't even let you try normal TTS unless you provide a reference voice so I noped out.
Re:A CC-By Open-Source TTS Model with Voice Cloning by miki123211 on Saturday November 09, 2024 @07:49AM (#42094148)
From what I'm seeing, most of the open source TTS models are trained on the same few voices, mostly in 16Khz, mostly from Librivox books I think.

Eleven Labs is most likely trained on stolen audiobooks, they've published a few Youtube videos in Polish, now taken down, of AI renditions of famous Polish audiobook narrators. This was all before they became popular, and before their voice cloning models were publicly available I think.

Re:A CC-By Open-Source TTS Model with Voice Cloning by generalizations on Saturday November 09, 2024 @11:00AM (#42095088)
> mostly from Librivox books

That probably explains a lot. I've tried listening to some of those audiobooks - very hit and miss, mostly miss. Definitely amateur hour and mostly bad quality.

Re:A CC-By Open-Source TTS Model with Voice Cloning by sandreas on Saturday November 09, 2024 @09:12AM (#42094513)
I had pretty good results with coqui-tts and a VITS model, I trained myself with an open dataset and later with one I extracted from audiobooks / epub and therefore can't publish (german)

The dataset and video tutorials are all available and linked on (also english):

https://www.thorsten-voice.de/en/motivation-vision/

Re:A CC-By Open-Source TTS Model with Voice Cloning by thorsten-voice on Sunday November 10, 2024 @07:05AM (#42099759)
Thanks for mentioning my Thorsten-Voice project, dear sandreas :)
Re:A CC-By Open-Source TTS Model with Voice Cloning by sandreas on Sunday November 10, 2024 @12:44PM (#42101423)
You're very welcome.