The usefulness is why the term is so widespread in familiarity but I think the term would have existed to describe the linguistic mapping even if they hadn't proven to have direct problem-solving capabilities.

That's an inductive bias, not an emergent property. The inductive bias of transformers is that they're good at integrating global context from different parts of a sequence without a particular bias towards recent time steps or localized regularities. And that happens to be a good fit for many (but not all) real-world sequence learning tasks.

The "emergent property" aspect is when LLMs are good at a task at scale X*3 but were incompetent at scale X.

My point was that this particular inductive bias doesn't inherently beget a "language model".

It does beget a Transformer which we choose to call a language model when it's applied to language data