Re:Show HN: AI Dub Tool I Made to Watch Foreign Language Vid...
by thylacine222 on Wednesday February 28, 2024 @01:34AM
(#39534761)
Whisper can also generate timings at the word level, which you could use to make better-timed subtitles
Re:Show HN: AI Dub Tool I Made to Watch Foreign Language Vid...
by leobg on Wednesday February 28, 2024 @03:25AM
(#39535434)
Yes. But Whisper's word-level timings are actually quite inaccurate out of the box. There are some Python libraries that mitigate that. I tested several of them. whisper-timestamped seems to be the best one. [0]
He talked about subtitling at some point, and I was surprised how cheap subtitling services are. I think he went beyond the price he mentioned, but it really made me question why big, profitable YouTube channels aren't spending the small change to do at least native language subtitles that Google can translate, instead of relying on YouTube's terrible algorithm
That said, Whisper seems to generate quite good subtitles that take short pauses for timing into account, but they're obviously neve going to be as good as a human that actually understands the context of what's being said.