Re:What it Sounds Like When Doves Cry: transcribing bird sou...
by og_kalu on Monday April 03, 2023 @06:11PM
(#35432025)
It's the latter.
That's why it can only be trained on 1.8% french by word count and speak fluent french. It's not that the amount of french that was in its corpus would have been enough to train it to that equivalent level alone (it wouldn't, not by a long shot). It's that there's positive transfer of knowledge. whatever it learns predicting english helps it learn other languages much faster. it's not the only display of positive transfer either. llms trained on code reason better.
It's also why the open ai instruct tuning dataset was almost entirely english yet it follows instruction in other languages just fine.
If there's something to be translated, a language model trained on both will translate it. it just needs to learn the concept of translation but the human side of the dataset should take care of that
Really i only say theoretically because of 2 things
one. first, a multimodal text-audio model that shows positive transfer between text and audio needs to be demonstrated. otherwise, this hypothetical human-dolphin model might need too much data to be feasible.
2. we're going to need a lot of dolphin speech audio.
That's why it can only be trained on 1.8% french by word count and speak fluent french. It's not that the amount of french that was in its corpus would have been enough to train it to that equivalent level alone (it wouldn't, not by a long shot). It's that there's positive transfer of knowledge. whatever it learns predicting english helps it learn other languages much faster. it's not the only display of positive transfer either. llms trained on code reason better.
It's also why the open ai instruct tuning dataset was almost entirely english yet it follows instruction in other languages just fine.
If there's something to be translated, a language model trained on both will translate it. it just needs to learn the concept of translation but the human side of the dataset should take care of that
Really i only say theoretically because of 2 things
one. first, a multimodal text-audio model that shows positive transfer between text and audio needs to be demonstrated. otherwise, this hypothetical human-dolphin model might need too much data to be feasible.
2. we're going to need a lot of dolphin speech audio.