But I'm not clear on whether that's because it's been trained on a bunch of aligned translations and bilingual dictionaries and whatnot, or if it's detecting the same repeating patterns in languages and figuring out the translation mappings on its own.

Is it the latter? Could a ChatGPT actually translate something like dolphin communication into semi-meaningful English, that wouldn't just be entirely nonsense?

That's why it can only be trained on 1.8% french by word count and speak fluent french. It's not that the amount of french that was in its corpus would have been enough to train it to that equivalent level alone (it wouldn't, not by a long shot). It's that there's positive transfer of knowledge. whatever it learns predicting english helps it learn other languages much faster. it's not the only display of positive transfer either. llms trained on code reason better.

It's also why the open ai instruct tuning dataset was almost entirely english yet it follows instruction in other languages just fine.

If there's something to be translated, a language model trained on both will translate it. it just needs to learn the concept of translation but the human side of the dataset should take care of that

Really i only say theoretically because of 2 things

one. first, a multimodal text-audio model that shows positive transfer between text and audio needs to be demonstrated. otherwise, this hypothetical human-dolphin model might need too much data to be feasible.

2. we're going to need a lot of dolphin speech audio.