Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by TOMDM on Sunday November 19, 2023 @06:37PM
(#38339848)
The community upgrades to whisper are far faster than real-time, especially if you have a powerful gpu
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by Jach on Monday November 20, 2023 @07:38AM
(#38347114)
There's several faster ones out there. I've been using https://github.com/Softcatala/whisper-ctranslate2 which includes a nice --live_transcribe flag. It's not as good as running it on a complete file but it's been helpful to get the gist of foreign language live streams.
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by wahnfrieden on Sunday November 19, 2023 @06:45PM
(#38339932)
use whisper-distil, it's like 5-8x faster
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by modeless on Sunday November 19, 2023 @08:27PM
(#38340938)
Great question! Whisper processes audio in 30 second chunks. But on a fast GPU it can finish in only 100 milliseconds or so. So you can run it 10+ times per second and get around 100ms latency. Even better actually because Whisper will predict past the end of the audio sometimes.
Isn't it quite non-realtime?