Funny how you said "it turns out" when the whole point is that we don't understand LLMs - we just empirically see what they are good at.

Claiming that you understand LLMs is similar to saying that you understand how our biology work because you understand evolution. No - you understand the mechanism behind evolution, but not the complexity it produces.