Re:BASE TTS: The largest text-to-speech model to-date
by qwertox on Wednesday February 14, 2024 @05:04PM
(#39376199)
I'm using Whisper to transcribe notes I record with a lavalier mic during my bike rides (wind is no problem), but am using OpenAI's service. When it was released I tested it on a Ryzen 5950x and it was too slow and memory hungry for my taste. Using large was necessary for that use case (also, I'm recording in German).
Re:BASE TTS: The largest text-to-speech model to-date
by GaggiX on Wednesday February 14, 2024 @05:55PM
(#39376743)
With Whisper, you can find many smaller models that are fine-tuned for a particular language, so even smaller models can perform adequately.
Re:BASE TTS: The largest text-to-speech model to-date
by kkielhofner on Thursday February 15, 2024 @04:49AM
(#39380812)
The original release was full precision model weights running in an old version of PyTorch with no optimizations.
Fast forward to now and you have faster-whisper (using Ctranslate2) and distil-whisper optimized weights.
Between the two of them Whisper Large uses something like 1/8th the memory and is likely at least an order of magnitude faster on your hardware.
German has no effect on these metrics and for accuracy it actually has a lower word error rate than English.
Re:BASE TTS: The largest text-to-speech model to-date
by jamil7 on Thursday February 15, 2024 @01:52AM
(#39379845)
Whisper is for STT though right?
Re:BASE TTS: The largest text-to-speech model to-date
by qwertox on Thursday February 15, 2024 @04:14AM
(#39380605)
The term STT is not used, it's called ASR, Automatic Speech Recognition. I mean, I was referring to both TTS and ASR in my comment.
I am running the smaller models in near real-time on a 3rd gen i7, with good results even using my terrible built-in laptop mic from a distance. The medium and large models are impressively accurate for technical language.