Re:Show HN: AI Dub Tool I Made to Watch Foreign Language Vid...
by postexitus on Wednesday February 28, 2024 @04:34AM
(#39535828)
Unfortunately not all languages follow the same sentence structure, so translating "up to what has been said so far" is not possible.
Assume 2 dramatic stops in an English sentence, and observe Turkish version. You can "I will.. go to.... the cinema" "Ben... sinemaya... gidecegim" (I .. to the cinema.. go)
I am sure there are smarter examples.
With the SRT file format for subtitles, I think, there's no reason why one couldn't make groups of words appear as they are spoken.
Actually, I have to do the same thing when generating the dubbed voices. Otherwise it feels as though the AI voice is saying something different than the person in the video, especially when the AI finishes speaking and you still hear some of the last words from the original speaker.