There’s a huge network effect at work here. It’s easier to find D&D players than players of any other game — not that you can’t find them, but D&D is easier. WotC runs organized play programs such that I can walk into almost any brick and mortar tabletop gaming store in the country and immediately find a regular game.
Look at https://startplaying.games/search and count the number of D&D games compared to anything else.
And this has happened before. D&D 4e got rid of the OGL, which made people have these same conversations. This did create Pathfinder, but even cloning the earlier version of D&D didn’t prevent WotC from dominating except during the year or two between 4e and the current edition. Plus 4e played really differently, which won’t be the case for this new edition.
People still are. There's been tons of innovation over the past 20 years, and some of it even seeped into D&D and Warhammer.
But despite everything, D&D is still the big name everybody recognises. Maybe there doesn't need to be a standard, but there is, and it's D&D. Just like with Windows.
TTRPG rules are not tabletop game rules: they don't say what happens, GM does. TTRPG rules are a language that abstracts away trivia in a somewhat objective manner.
This is similar to having many programming languages: ideally you would want to have a single language for everything, but they all suck in different small ways.
I don't understand why there needs to be "a standard". in the 80's, 90's people played Runequest, Warhammer, GURPS, AD&D, Dark Eye... there was no need for a "standard". No, people are lazy and just want to play what is popular, this isn't a standard. Games rules are half the table top RPG, the stories are as important if not more...
Furthermore, back in the days, people were interested in innovative rules or universes, D&D is none of that, it's the most bland and boring game system...