That semantics is indeed useless.
And to begin with, applications should never be interested in all record pertaining to a domain; they want specific records. If I'm sending mail, I want a MX record for a domain. I don't want the SPF record unless I'm receiving mail which implicates that domain. If I'm making a HTTP connection, I don't want any of these; I want an IP address. So no semantics of ANY is easily justifiable.
On the other hand, we can't claim that this semantics is hard to implement.
It's hard to implement in a way that makes sense in the constraints of the world as it is (people asking for ANY when they have some meaning in mind other than the nonsensical specs) and especially in a way that doesn't enable DNS reflection attacks. In an ideal world, sure, you could enable ANY provided nobody either good or evil ever used it. And in fact Cloudflare says they did previously implement it just fine, by littering if statements everywhere.
However, also, pre-RFC8482 that's not what ANY is defined to mean. A non-authoritative caching resolver is permitted to respond with only the records it has cached if something is in cache. So you're already getting a random subset of records. So the behavior is unhelpful for clients and also unhelpful for authoritative servers, and not in a way where the unhelpfulness trades off for helpfulness elsewhere.