News for hackers. Stuff that matters.
Re:AI could end foreign-language subtitles by aeturnum on Wednesday January 19, 2022 @06:58PM (#30002284)
I would also love to a more responsive subtitle format! I've seen some really wonderful custom jobs where people use different subtitle colors for different speakers, place subtitles near the speaker in the frame, etc. I think there is a ton of room for improving the expressiveness of the medium.
Re:AI could end foreign-language subtitles by marcan_42 on Wednesday January 19, 2022 @09:29PM (#30003637)
The format exists, it's called ASS. It can do things like this:

https://youtu.be/djrJP_Pi03U

https://youtu.be/R3shsgKbd_M?t=12s

That's a pure text format; no bitmaps involved, just markup and TTF fonts. It can do vector shapes and perspective positioning. These are somewhat overdone examples, but one more common nice use case is literally translating signs on buildings, objects and such, seamlessly, while preserving the image style. The whole thing is rendered in real time and you can toggle the subs off and see the original frame.

https://youtu.be/n_c-eKTisI0

Again, that's all markup rendered on the video in real time from fonts. Yes, really.

It is the standard in the anime fansub community, but has seen little adoption elsewhere. Partially this is because it seems commercial operations rarely have much interest in subtitle quality or technology.

Incidentally, even YouTube supports an undocumented subtitle format that can support a decent amount of styling (YTT). Unfortunately, support is inconsistent across web/mobile versions, and it looks best when viewed in a browser. I've used it to do things like this (turn on captions):

https://www.youtube.com/watch?v=dh1BCyjCbes

https://www.youtube.com/watch?v=IxLI7KmzOjE

You can convert a subset of ASS to that format with this tool: https://github.com/arcusmaximus/YTSubConverter

Re:AI could end foreign-language subtitles by aeturnum on Wednesday January 19, 2022 @10:11PM (#30004008)
Oh wow! That is amazing. The examples I was describing were in anime so I have no doubt they were in ASS as well. Very cool.
Re:AI could end foreign-language subtitles by zeroimpl on Wednesday January 19, 2022 @08:42PM (#30003189)
A pet peeve of mine has been subtitles which flip the order of words, particularly when the sentence is broken into two lines. Like the spoken text in foreign language is “A murderer is... Joe” but it gets translated to “Joe is a... murderer”.

The other day I happened to see live content with closed captions, and I thought I would love the option to see subtitles like this, where words are translated precisely when spoken.

Re:AI could end foreign-language subtitles by shkkmo on Wednesday January 19, 2022 @09:24PM (#30003596)
But translation doesn't really work like that. There are significant tradeoffs between verbosity, cadence, meaning and subtext when translating. The different contraints are why the forieng dubbibg and foreign subtitles are usually quite different.

You generally only see word for word matches with subtitles when the subtitles are in the same language as the original lines spoken by the actors.

Re:AI could end foreign-language subtitles by lisper on Thursday January 20, 2022 @12:34AM (#30005110)
And the GP's example is the perfect illustration. "A murderer is Joe" sounds really weird. No native English speaker would ever say that. They would say, "Joe is a murderer" or "The murderer is Joe" or even "One of the murderers is Joe." But never "A murderer is Joe." Maybe in a Victorian novel, but not in a movie.
Re:AI could end foreign-language subtitles by zeroimpl on Thursday January 20, 2022 @10:03AM (#30009720)
There are certainly tradeoffs, which is why I said I’d like the option.

Since I’m moderately familiar with the other language, I really just need the translation for words I don’t understand or mis-hear.

There is such a thing as a real-time translation. I’ve heard it in news broadcasts when a politician speaks a foreign language. It also happens all the time at the UN and other international negotiations. It works well but has downsides such as occasional latency while the translator needs to hear a few more words before they can translate.

Re:AI could end foreign-language subtitles by shkkmo on Saturday January 22, 2022 @10:48AM (#30037640)
I don't think real time translation matches word for word either.

Something that might be sort of like what you are looking for is Language Learning with Netflix https://chrome.google.com/webstore/detail/language-learning-...

It's a browser plugin that provides advanced subtitle functionality. You can have dusl sets of subtitles up, or just the original language subtitles with the option to pause and mouse over individual words for the definition.

Re:AI could end foreign-language subtitles by cyphar on Thursday January 20, 2022 @07:55AM (#30008276)
Unfortunately it's difficult to translate sentences between languages in a way that preserves word order and is grammatically correct in the translated language (your example is a pretty good example of this).

Different languages have very different grammar, and even the best translated subtitles won't be able to handle a clever usage of the original language's grammar.

This is one of the many tradeoffs you make with translated works. You always lose something (sometimes a lot) in the translation.