It’s not really a question of high resource vs low resource languages so much as what languages ended up in the training corpus.
1. There’s a lot of transfer learning going on with predict the next token LLMs. A model trained on 500B tokens of English and 50b tokens of French will speak French far better than if it was trained on only 50b tokens of French.
2. You don’t need parallel corpora for every single pair you want to translate to. This means that GPT LLMs only need single text data for the vast majority of languages. Training most NMT models you would need Burmese/English parallel data.
Both of the above combine to mean that not only is quality demonstrably better, amount of data needed is lower too.
GPT's burmese isn't worse because it's low resource. It's because open ai made no specific attempt to include burmese text.
They're not even trying. GPT-3 was 93% English with the 2nd biggest holder less than 2%
Neural Machine Translation models don't need those either. That's been the case at least since 2016.
That was one of the big claims by Google, that Google Translate had created its own "interlingua" thanks to which it could translate between languages for which it doesn't have parallel corpora:
Following several months of testing, the researchers behind the AI have seen it be able to blindly translate languages even if it's never studied one of the languages involved in the translation. "An example of this would be translations between Korean and Japanese where Korean⇄Japanese examples were not shown to the system," the Mike Schuster, from Google Brain wrote in a blogpost.
https://www.wired.co.uk/article/google-ai-language-create
It was obvious already back then that this "interlingua" was no other than English, e.g., from the same article:
The team said the system was able to make "reasonable" translations of the languages it had not been taught to translate. In one instance, a research paper published alongside the blog, says the AI was taught Portuguese→English and English→Spanish. It was then able to make translations between Portuguese→Spanish.
But, hey, hype.
Right. But those models still operate on parallel corpora alone.
In LLMs, translation pairs are not the only data improving translations.
>But, hey, hype
It's not hype man. GPT-4 is much better for distant language pairs.
Please translate this:
ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။
The sentence "ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။" translates to "They gave me a second chance." in English.
ChatGPT translates ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။ as "I have received a second warning.", which is incorrect. အခွင့် https://en.wiktionary.org/wiki/%E1%80%A1%E1%80%81%E1%80%BD%E... does not mean "warning", even though that is a likely completion of "I have received a second " in English.
Google Translate gives me "Gave me a second chance.", which closely matches the Burmese sentence down to dropping the subject (common in Burmese, rare in English) which makes the translation sound weird.
So any claim that ChatGPT is better/worse at translating really needs to specify the languages involved and what your goal for the translation is. (E.g. the benchmark paper you link seems to focus on the ability to steer the translation by providing additional context.)