To be fair, this is an infamous failure mode of neural mtl too, and a big part of what makes the discourse around GPT so ... evocative of the discourse in 2017.
I understand Korean. I've tested it there too and my evaluations haven't changed. It's better. A lot better.
That aside, nothing you've said here doesn't apply to NMT.
I'm going to be Frank here. You don't what you're talking about when it comes NMT and it makes this conversation very frustrating. You keep mentioning cons that apply to NMT as well.
Literally almost all SOTA NMT models (NLLB for instance) today are Transformers or some transformer variant (In fact this was the original proposition in the famous Attention Paper). They're just transformers trained for a translation primary objective.
Those cons of GPT, NMT inherit them too. They hallucinate, they can make pseudo translations etc.
>EDIT: Oh wow, all of your 107 submissions to HN from the 6 months your account has existed, have been about AI and LLMs.
So ? That's what I like to do. Is this some attempt at an ad hominem?
The 2nd paper I linked is about document level translations.
>The entire industry hasn't released an LLM solution yet for translation (excepting the rewording mini features). What makes you think you know more than those on the ground?
Training and inference for GPT scale models is extremely expensive.
Dude, don't be an asshole. You got useful experience. Help others with it instead of putting them down.
But I guess that would indeed sound much better around beers.
Translator for 20 years talking about using GPT for Japanese-English.
It may present more fluent text, but if it doesn't know it's strayed from the source text and you can't tell either (because you don't understand source language) then you'll end up with error-laden pseudo translations. At least with NMT you know the errors are consistent.
I don't know who you know who thinks GPT is ahead, but nobody in the very well funded translation industry has a GPT powered translation engine for the key reason that it's not ready for production. For a human to post edit MT, we're mainly talking fixing broken vocab. You'd never present raw MT to a client. It needs editing. Heavily. To think LLM translation doesn't need editing is either coming from someone not in the industry, or blinded by hype. And the kind of editing issues are more insidious, like those found in voice dictated texts. Homophones aren't flagged by QA software because they are real words. Just like LLMs make real sentences, except when they don't, but good luck detecting that, and editing out the additional meaning it has decided to inject.
Have you tried to run the GPT-4 API on a segmented xliff at all? If the segmentation is bad, and full of tags, GPT4 breaks completely. It tries to close sentences that run across segments, it can't handle tags in-line (the chatgpt interface can, but you can't use that at scale).
It can do some impressive work, don't get me wrong, but I'm not sure how hands-on you've really been if you think it's a solved problem.
Production translation is a non-trivial output. The entire industry hasn't released an LLM solution yet for translation (excepting the rewording mini features). What makes you think you know more than those on the ground? Or have you developed something that's still in stealth?
EDIT: Oh wow, all of your 107 submissions to HN from the 6 months your account has existed, have been about AI and LLMs. I guess I got the hype part right. As for industry knowledge the jury is still out, but this could well be the classic HN "I understand tech so obviously I understand everything" play. Keep us posted!