This doesn't just apply to grammars. There is a huge array of formalisms (e.g. logics, type systems) out there and most just look like the result of someone saying "what if I did this?".
It's alright- if I'm being naive, I'm being naive.
But- what am I missing? You're saying we're doing it wrong- how? For me the intuition that infinite generative ability flows naturally from unbounded recursion, like an egg from a hen's bottom, is kind of obvious. Is it naive? I guess it's empirical, for me at least.
Also, btw, I was introduced to the idea of language equivalence through Hopcroft and Ullman, so from the point of view of computer science, where it's been very useful, in practical terms. I guess if you're coming from a mathematical or theoretical physics background it might sound a bit silly, but it's allowed us to make a lot of progress, for instance to create a few thousand different architectures and languages... but maybe I shouldn't be bringing that up as progress...
Anyway, I don't know- how would you interpret the observation correctly? Where are we going wrong?
Look, no. Grammars look like "the key to the gates" because of the equivalence between grammars-language-automata etc. You can write up grammars for fully Turing-complete languages, or indeed context-free languages that allow you to declare Turing-complete automata.
You can create grammars that display infinite recursion with minimal effort:
So it's not like someone (cough, Chomsky) woke up one nice day and thought "blimey, I'll tell the world that grammars are a powerful tool for modelling hierarchical processes". It's that they are.We have used grammars in practice to model complex hierarchical processes- except it's only those processes that we already know how to model, because we came up with them ourselves, like the aforementioned Turing-complete ones.
The problems begin when we try to fit a process we hardly understand to a grammar. That is not a limitation of the tool itself. It's a limitation of our ability to use it.