But whales? What we are (probably) proposing is putting two languages side by side with NO context and saying "figure it out".
Here is a great example of people and language and perception: https://news.mit.edu/2023/how-blue-and-green-appeared-langua...
Im fairly confident that unless we give an LLM a LOT of context between data sets to work with that it won't make much progress at all.
while ww2 code breaking contained other human contexts, perhaps like-context observations could allow for certain categories that might reveal linguistic insights of whales. like whale sounds observed consistently near boats, divers, danger, eating, play, etc.
no clue about how LLM/AI systems process meta contextual training data inputs (is this even a valid phrase about AI?! haha) but i hope whale linguists have considered such an approach.
1. Many such concepts cannot be directly mapped between different languages especially with distant language pairs.
2. Language Models and Image models even if both are only trained on their respective modality learn structural representations so similar, you can connect them with a simple linear layer. That's it. Entirely different modalities https://arxiv.org/abs/2209.15162
https://arxiv.org/abs/2304.08485
I think you are severely underestimating the extent to which representations can group in neural networks.
They're mammals, they're social animals, they see. You're assuming a level of alienness we don't have the knowledge or understanding to truly ascertain.
You can do both. Models trained to receive and predict audio tokens alongside text are coming. You could just add such data as part of training.
>Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality.
If you trained it alongside human languages then shared context(if available) will be figured out by the model. They'll coalesce in the same shared space the way they do for human languages. Enough to translate without examples instead of just speak? Perhaps - it's not like there are examples in the training set for every lang to lang combination modern models are capable of translating.