I'm not sure I understand the analogy. The very idea of NNs is that it's not perfect, it is messy and not optimal, but is very generalizable.

>> The very idea of NNs is that it's not perfect, it is messy and not optimal, but is very generalizable.

Newton: Do you need more than that to describe the speed of a thrown baseball on a train? No. DO you you need more than newton to get to the moon? No. Is it going to be accurate at high speed in a large scale system (anything traveling near C)? NO, it fails spectacularly.

NN's are great at simulation, language, weather... But what people using them for weather seem to understand and the ML folks (screaming about AI and AGI) dont is that simulation is not a path to emulation. Lorenz showed that there were limits in weather, that most other disciplines have embraced these limits.

The entire innovation (discovery?) of LLMs is that a good formula for the task of sequence completion turns out to also be a good formula for a wide range of AI tasks. That emergent property is why language models are called language models.

The usefulness is why the term is so widespread in familiarity but I think the term would have existed to describe the linguistic mapping even if they hadn't proven to have direct problem-solving capabilities.

That's an inductive bias, not an emergent property. The inductive bias of transformers is that they're good at integrating global context from different parts of a sequence without a particular bias towards recent time steps or localized regularities. And that happens to be a good fit for many (but not all) real-world sequence learning tasks.

The "emergent property" aspect is when LLMs are good at a task at scale X*3 but were incompetent at scale X.

My point was that this particular inductive bias doesn't inherently beget a "language model".

It does beget a Transformer which we choose to call a language model when it's applied to language data