It just happens to be the most widespread by accident, because hyperlinked documents - HTML - became huge and then more and more UI elements were bolted on top of it. And suddenly it became the goto, because the plattform run everywhere.

But it is still a ugly mess underneath and you might be right, that it is the future, but it is not a great one. I hope a great one will surface one day and then we can start new with something sane.

"I'm excited for the new crop of browser engines because they could fix those deficiencies"

And I cannot see what fixing those deficiencies could mean other, than throwing most of the standard away. It doesn't mean all of it needs to be thrown away. WebAudio API, WebGPU, etc. all became great standards. But simple UI elements, like a slider or color picker are still just ugly and impossible to make beautiful and behave sane, besides making a new one by hand, or using a libary from someone who did that. But - with WebGPU especially - I am excited for the possibility to build a sane framework on top of some subsets of the browsers capabilities.