News for hackers. Stuff that matters.
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech by tomp on Sunday November 19, 2023 @06:30PM (#38339779)
How do you get Whisper to be fast?

Isn't it quite non-realtime?

Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech by TOMDM on Sunday November 19, 2023 @06:37PM (#38339848)
The community upgrades to whisper are far faster than real-time, especially if you have a powerful gpu
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech by Jach on Monday November 20, 2023 @07:38AM (#38347114)
There's several faster ones out there. I've been using https://github.com/Softcatala/whisper-ctranslate2 which includes a nice --live_transcribe flag. It's not as good as running it on a complete file but it's been helpful to get the gist of foreign language live streams.
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech by wahnfrieden on Sunday November 19, 2023 @06:45PM (#38339932)
use whisper-distil, it's like 5-8x faster
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech by modeless on Sunday November 19, 2023 @08:27PM (#38340938)
Great question! Whisper processes audio in 30 second chunks. But on a fast GPU it can finish in only 100 milliseconds or so. So you can run it 10+ times per second and get around 100ms latency. Even better actually because Whisper will predict past the end of the audio sometimes.