1) Changes to HTML can create security issues, especially if it results in a change to the HTML parser. You have to consider that non-browser use cases still need to be able to parse HTML correctly. Since we opted to make HTML versionless with <!doctype html> it became harder to do.
2) Big changes are harder to get consensus on. This is one of the reasons why they opt to do smaller, less controversial changes.
3) w3c and whatwg are dominated by large corporations, who hire thousands of engineers to work on their web properties. They are able to overcome the shortcomings of HTML easier than small teams and single-developer websites.
I agree though, this is a major problem. HTML is essentially a dead language, no major changes have been made in almost a decade.
W3C isn't interested in advancing HTML as anything and, even if they were, they'd be screaming into the night because none of the people who build browsers are interested in what they have to say. They’ve got their own club, and that's where HTML happens (and WHATWG and the browser vendors don't seem to be interested in pushing functionality into HTML in any big way, they are happy with the basic shape of the existing HTML/CS/JS trinity and don't see a big need to push for standalone HTML as a platform.)
The stupid part was advertising of writing XML by hand - both XHTML and XSLT. That is machine format - "application" in "application/xml". Not more sense than writing JS AST. And semantic web has no advantage for author.
XHTML 2.0 - XML Events is like IE <script for>, <img>alt</img> is good, <h> is good, removal of <i>, <b> - no one would see, XForms - partly adopted, and RDF - don't know.
Though it kind of understandable - XML was in rage. Everyone pushed it, Microsoft above all, IE5 accepted "text/xml" [1]. And Microsoft pushed XSLT without XSL-FO.
i knew there was some drama betweeen the two organziations but didn't know the history
I readily admit I may be missing something.
[1] - https://github.com/bigskysoftware/htmx/blob/master/src/htmx....
[2] - https://en.wikipedia.org/wiki/XHTML#XHTML_2.0